Systems and methods for determining translation accuracy in multi-user multi-lingual communications

ABSTRACT

Various embodiments described herein facilitate multi-lingual communications. The systems and methods of some embodiments enable multi-lingual communications through different modes of communication including, for example, Internet-based chat, e-mail, text-based mobile phone communications, postings to online forums, postings to online social media services, and the like. Certain embodiments implement communication systems and methods that translate text between two or more languages. Users of the systems and methods may be incentivized to submit corrections for inaccurate or erroneous translations, and may receive a reward for these submissions. Systems and methods for accessing the accuracy of translations using word based and language based features are described.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims priority from U.S.patent application Ser. No. 14/671,243, filed Mar. 27, 2015, which is acontinuation of and claims priority from U.S. patent application Ser.No. 14/294,668, filed Jun. 3, 2014, which is a continuation-in-part ofand claims priority from U.S. patent application Ser. No. 13/908,979,filed Jun. 3, 2013, entitled “Systems and Methods for Incentivizing UserFeedback for Translation Processing,” which is a continuation-in-part ofand claims priority from U.S. patent application Ser. No. 13/763,565,filed Feb. 8, 2013, entitled “Systems and Methods for Multi-UserMulti-Lingual Communications,” and claims priority from U.S. ProvisionalPatent Application Ser. No. 61/778,282, filed Mar. 12, 2013, entitled“Group Chat Translation Systems and Methods,” each of which isincorporated herein by reference.

FIELD OF THE INVENTION

The present invention(s) generally relate to language translation and,more particularly, language translation involving multiple users andmultiple languages.

BACKGROUND OF THE INVENTION

Before the advent of machine-based language translations (hereafter,“machine translations”), translation between two languages was onlypossible via intervention or interpretation by a person educated in bothlanguages. In contrast, typical machine translators generally operatebased on statistical/stochastic analysis of context and grammar, usuallywithout need of human intervention/interpretation.

Typical machine translation is often error prone, particularly where thetext to be translated has a minimal context. Text having minimal contextis often found in conversations, which employ brief sentenceconstruction. Additionally, machine translations often have trouble withabbreviations, acronyms, diminutives, colloquial words/phrases, propernouns, and common nouns, which are also commonly found in conversationaltext.

SUMMARY OF THE INVENTION

Various embodiments described herein provide for systems and methodsthat relate to multi-lingual communications between multiple users,possibly where the users are at two or more client systems. Modes ofcommunications facilitated by embodiments may include Internet-basedchat (e.g., Apple® iMessage, Windows® Live Messenger, etc.), e-mail(e.g., embedded forum messaging, Yahoo® mail, RFC 5322, etc.),text-based mobile phone communications (e.g., SMS messages or MMSmessages), postings to online forums (e.g., postings to a web-basedhobby forum), and postings to online social media services (e.g.,Twitter®, Facebook®, etc.). For example, systems and methods mayimplement a multi-lingual, multi-user chat system.

For some embodiments, the method provided comprises identifying a firstlanguage and a second language, receiving an initial message in thefirst language from a first person at a first chat client system whocommunicates in the first language, and querying a data store for afirst corresponding message, in the second language, that is based onthe initial message in the first language. If the data store includesthe first corresponding message, the method may then assist in sendingthe corresponding message to a second person at a second chat clientsystem who communicates in the second language. Depending on theembodiment, the initial message may comprise text, an emoticon.ASCII-based art, or other content suitable or customary for ahuman-readable message sent over a network. Additionally, the initialmessage may be part of a larger message being communicated between chatclient systems (e.g., the initial message is one sentence in amulti-sentence message).

If the data store does not include the first corresponding message, themethod may utilize a transformation engine to attempt to transform atleast a portion of the initial message to a transformed message in thefirst language. Using the transformed message, the method may then querythe data store for a second corresponding message, in the secondlanguage, that is based on the transformed message.

For certain embodiments, the system or method may attempt transformingthe initial message using a series of transformation operations beforequerying the data store is queried for a second corresponding messagethat is based on the transformed message. Alternatively, in someembodiments, the system or method may perform the transformation andquery iteratively, whereby the initial message is transformed using asubset of available transformation operations, the data store is queriedfor a second corresponding message based on the resulting transformedmessage, and if a second corresponding message is not identified,another iteration of transformation and query is performed (e.g., theresulting transformed message is further transformed using anothersubset available transformation operations, and the data store isqueried for a second corresponding message based on the resultingtransformed message). In some such embodiments, the subset oftransformation operations applied in each iteration may be applied tothe initial message or may be applied to the latest resultingtransformed message.

Eventually, the method may assist in translating the initial message orthe transformed message to a corresponding message in the secondlanguage. In some embodiments, the initial message may be translated tothe corresponding message when the first corresponding message for theinitial message is not in the data store and the transformation enginedoes not transform at least a portion of the initial message.Additionally, in various embodiments, the transformed message may betranslated to the corresponding message when: the first correspondingmessage for the initial message is not in the data store; thetransformation engine results in a transformed message that contains thetransformation of at least a portion of the initial message; and thedata store does not include the second corresponding message for thetransformed message.

Depending on the embodiment, transforming the portion of the initialmessage may comprise identifying a chatspeak word or phrase (e.g.,‘lol,’ ‘gr8’) in the initial message and replacing the chatspeak word orphrase with a non-chatspeak word or phrase, performing a spelling checkon the portion of the initial message, or identifying an abbreviation inthe portion of the initial message and replacing the abbreviation with aword or a phrase corresponding to (e.g., represented by) theabbreviation (e.g., ‘CA’ with ‘California,’ or ‘brb’ to ‘be rightback’).

In addition, transforming the portion of the initial message maycomprise identifying an acronym in the portion of the initial messageand replacing the acronym with a word or a phrase corresponding to(e.g., represented by) the acronym (e.g., ‘USA’), or identifying acolloquial word or phrase in the portion of the initial message andreplacing the colloquial word or phrase with a word or a phraserepresenting the colloquial word or phrase. Furthermore, transformingthe portion of the initial message may comprise identifying a profaneword or phrase in the portion of the initial message and replacing theprofane word or phrase with a non-profane word or a phrase (e.g., thatis representative of the profane word or phrase) or removing the profaneword or phrase from the initial message.

For some embodiments, transforming the portion of the initial messagecomprises flagging the portion of the initial message to not betranslated. For instance, wherein a certain portion of the initialmessage comprises a proper noun, a common noun, a diminutive, anabbreviation, or an acronym, the method may flag that certain portionsuch that it is not translated in subsequent operations.

Certain embodiments provide for a system comprising various componentsthat are configured to perform various operations described herein.Likewise, certain embodiments provides for a computer program productcomprising computer instruction codes configured to cause the computersystem to perform various operations described herein.

In one aspect, the invention relates to a computer-implemented method.The method includes: selecting from a data store a word or phraseassociated with a failure to translate a message containing the word orphrase from a first language to a second language; selecting a user fromwhich to solicit user feedback for the translation failure; determininga value of an incentive to offer the user in exchange for the userfeedback; sending a request for the feedback to a computing device ofthe user, the request including the incentive; receiving the userfeedback from the computing device wherein the user feedback includes arespective word or phrase in the first or second language; determiningthat the user feedback is approved; and based on the approval, creditingan account of the user according to the value of the incentive.

In certain embodiments, the failure is due to an actual failure totranslate the message. The failure may be identified by or may be due toa user flagging the message as potentially incorrect. Selecting the usermay be based on a confidence measure of the user, a quota associatedwith the user, a previous credit to the account of the user, apreference of the user, or a language ability of the user. In someembodiments, the word or phrase includes chatspeak in the firstlanguage. The response may include chatspeak in the second language. Insome implementations, the query includes a field configured to receive atext-based value.

In certain embodiments, the request includes a set of preselecteddefinitions from which the user can choose a definition for the word orphrase. The set of preselected definitions may include, for example, atleast one definition provided by another user in response to anotherrequest, the other request being previously generated to obtain previoususer feedback for the word or phrase from the other user. The otherrequest may include another set of preselected definitions from whichthe other user chose the definition. The method may also includeevaluating the user feedback to determine a most popular response.

In various embodiments, the method also includes determining acompetency of the user based on the user feedback. The method may alsoinclude updating a transformation or translation of the word or phrasefrom the first language to the second language based on the userfeedback. In some embodiments, determining that the user feedback isapproved may include determining that the user feedback is notfraudulent and/or determining that the user feedback is accurate.Determining that the user feedback is approved may be based on acomparison of the user feedback to at least one previous user feedbackprovided by another user in response to another request, the otherrequest being previously generated to obtain feedback for the word orphrase from the other user.

In certain embodiments, the incentive includes (or is an offer for)in-game currency or an in-game item. The value of the incentive may bedetermined based on, for example, a complexity of the word or phrase orimportance of the word or phrase. In some examples, determining thevalue of the incentive includes considering (i) a complexity of the wordor phrase, (ii) an importance of the word or phrase, (iii) a responsemethod employed by the user, (iv) a type of word or phrase, and/or (v) alanguage involved in the translation failure.

In another aspect, the invention relates to a system that includes oneor more computers programmed to perform operations. The operationsinclude: selecting from a data store a word or phrase associated with afailure to translate a message containing the word or phrase from afirst language to a second language; selecting a user from which tosolicit user feedback for the translation failure; determining a valueof an incentive to offer the user in exchange for the user feedback;sending a request for the feedback to a computing device of the user,the request including the incentive; receiving the user feedback fromthe computing device wherein the user feedback includes a respectiveword or phrase in the first or second language; determining that theuser feedback is approved; and based on the approval, crediting anaccount of the user.

In certain embodiments, the failure is due to an actual failure totranslate the message. The failure may be identified by or may be due toa user flagging the message as potentially incorrect. Selecting the usermay be based on a confidence measure of the user, a quota associatedwith the user, a previous credit to the account of the user, apreference of the user, or a language ability of the user. In someembodiments, the word or phrase includes chatspeak in the firstlanguage. The response may include chatspeak in the second language. Insome implementations, the query includes a field configured to receive atext-based value.

In certain embodiments, the request includes a set of preselecteddefinitions from which the user can choose a definition for the word orphrase. The set of preselected definitions may include, for example, atleast one definition provided by another user in response to anotherrequest, the other request being previously generated to obtain previoususer feedback for the word or phrase from the other user. The otherrequest may include another set of preselected definitions from whichthe other user chose the definition. The operations may also includeevaluating the user feedback to determine a most popular response.

In various embodiments, the operations also include determining acompetency of the user based on the user feedback. The operations mayalso include updating a transformation or translation of the word orphrase from the first language to the second language based on the userfeedback. In some embodiments, determining that the user feedback isapproved may include determining that the user feedback is notfraudulent and/or determining that the user feedback is accurate.Determining that the user feedback is approved may be based on acomparison of the user feedback to at least one previous user feedbackprovided by another user in response to another request, the otherrequest being previously generated to obtain feedback for the word orphrase from the other user.

In certain embodiments, the incentive includes (or is an offer for)in-game currency or an in-game item. The value of the incentive may bedetermined based on, for example, a complexity of the word or phrase orimportance of the word or phrase. In some examples, determining thevalue of the incentive includes considering (i) a complexity of the wordor phrase, (ii) an importance of the word or phrase, (iii) a responsemethod employed by the user, (iv) a type of word or phrase, and/or (v) alanguage involved in the translation failure.

In another aspect, the invention relates to a computer program productstored in one or more storage media for improving language translationthrough incentivized feedback. The computer program product isexecutable by the data processing apparatus to cause the data processingapparatus to perform operations that include: selecting from a datastore a word or phrase associated with a failure to translate a messagecontaining the word or phrase from a first language to a secondlanguage; selecting a user from which to solicit user feedback for thetranslation failure; determining a value of an incentive to offer theuser in exchange for the user feedback; sending a request for thefeedback to a computing device of the user, the request including theincentive; receiving the user feedback from the computing device whereinthe user feedback includes a respective word or phrase in the first orsecond language; determining that the user feedback is approved; andbased on the approval, crediting an account of the user.

In certain embodiments, the failure is due to an actual failure totranslate the message. The failure may be identified by or may be due toa user flagging the message as potentially incorrect. Selecting the usermay be based on a confidence measure of the user, a quota associatedwith the user, a previous credit to the account of the user, apreference of the user, or a language ability of the user. In someembodiments, the word or phrase includes chatspeak in the firstlanguage. The response may include chatspeak in the second language. Insome implementations, the query includes a field configured to receive atext-based value.

In certain embodiments, the request includes a set of preselecteddefinitions from which the user can choose a definition for the word orphrase. The set of preselected definitions may include, for example, atleast one definition provided by another user in response to anotherrequest, the other request being previously generated to obtain previoususer feedback for the word or phrase from the other user. The otherrequest may include another set of preselected definitions from whichthe other user chose the definition. The operations may also includeevaluating the user feedback to determine a most popular response.

In various embodiments, the operations also include determining acompetency of the user based on the user feedback. The operations mayalso include updating a transformation or translation of the word orphrase from the first language to the second language based on the userfeedback. In some embodiments, determining that the user feedback isapproved may include determining that the user feedback is notfraudulent and/or determining that the user feedback is accurate.Determining that the user feedback is approved may be based on acomparison of the user feedback to at least one previous user feedbackprovided by another user in response to another request, the otherrequest being previously generated to obtain feedback for the word orphrase from the other user.

In certain embodiments, the incentive includes (or is an offer for)in-game currency or an in-game item. The value of the incentive may bedetermined based on, for example, a complexity of the word or phrase orimportance of the word or phrase. In some examples, determining thevalue of the incentive includes considering (i) a complexity of the wordor phrase, (ii) an importance of the word or phrase, (iii) a responsemethod employed by the user, (iv) a type of word or phrase, and/or (v) alanguage involved in the translation failure.

In one aspect, the invention relates to a method implemented by a dataprocessing apparatus. The method includes: providing a text message chatsystem to a plurality of users; receiving an original text message in afirst language from a first user; generating an initial translation in asecond language of the original text message; providing the originaltext message and the initial translation to a second user; receiving atranslation correction from the second user to address an error in theinitial translation; and at least one of: (a) identifying a mostaccurate translation correction from a plurality of translationcorrections, the plurality of translation corrections including thetranslation correction from the second user; and (b) evaluating anaccuracy of the translation correction from the second user using aword-based feature, a language-based feature, and/or a word alignmentfeature.

In certain embodiments, the method includes offering an incentive (e.g.,a virtual good and/or a virtual currency for use in an online game) toencourage the second user to submit the translation correction.Determining the most accurate translation correction may include:receiving at least one additional translation correction from at leastone additional user to address the error in the initial translation,wherein the at least one additional translation correction and thetranslation correction from the second user define the plurality oftranslation corrections; receiving feedback from users regarding anaccuracy of the plurality of translation corrections; and, based on thefeedback, identifying the most accurate translation correction from theplurality of translation corrections.

In some implementations, the method also includes providing a reward(e.g., a virtual good and/or a virtual currency for use in an onlinegame) to a user who submitted the most accurate translation correction.The method may also include providing a reward (e.g., a virtual goodand/or a virtual currency for use in an online game) to a user whoprovided the feedback used to identify the most accurate translation.The word-based feature may include, for example, a word count, acharacter count, an emojis, a number, and/or a punctuation mark. Usingthe language-based feature may include identifying parts of speechpresent in the original text message and in the translation correctionfrom the second user.

In some embodiments, the method also includes: identifying a number ofverbs present in each of the original text message and the translationcorrection from the second user; and comparing the number of verbs inthe original text message with the number of verbs in the translationcorrection from the second user. An absence of a part of speech in theoriginal text message and/or the translation correction from the seconduser may be indicative of a language detection failure. The method mayalso include rejecting the translation correction from the second userwhen the translation correction from the second user is the same as theinitial translation.

In another aspect, the invention relates to a system that includes acomputer readable medium having instructions stored thereon, and a dataprocessing apparatus. The data processing apparatus is configured toexecute the instructions to perform operations including: providing atext message chat system to a plurality of users; receiving an originaltext message in a first language from a first user; generating aninitial translation in a second language of the original text message;providing the original text message and the initial translation to asecond user; receiving a translation correction from the second user toaddress an error in the initial translation; and at least one of: (a)identifying a most accurate translation correction from a plurality oftranslation corrections, the plurality of translation correctionsincluding the translation correction from the second user; and (b)evaluating an accuracy of the translation correction from the seconduser using a word-based feature, a language-based feature, and/or a wordalignment feature.

In certain embodiments, the operations include offering an incentive(e.g., a virtual good and/or a virtual currency for use in an onlinegame) to encourage the second user to submit the translation correction.Determining the most accurate translation correction may include:receiving at least one additional translation correction from at leastone additional user to address the error in the initial translation,wherein the at least one additional translation correction and thetranslation correction from the second user define the plurality oftranslation corrections; receiving feedback from users regarding anaccuracy of the plurality of translation corrections; and, based on thefeedback, identifying the most accurate translation correction from theplurality of translation corrections.

In some implementations, the operations also include providing a reward(e.g., a virtual good and/or a virtual currency for use in an onlinegame) to a user who submitted the most accurate translation correction.The operations may also include providing a reward (e.g., a virtual goodand/or a virtual currency for use in an online game) to a user whoprovided the feedback used to identify the most accurate translation.The word-based feature may include, for example, a word count, acharacter count, an emojis, a number, and/or a punctuation mark. Usingthe language-based feature may include identifying parts of speechpresent in the original text message and in the translation correctionfrom the second user.

In some embodiments, the operations also include: identifying a numberof verbs present in each of the original text message and thetranslation correction from the second user; and comparing the number ofverbs in the original text message with the number of verbs in thetranslation correction from the second user. An absence of a part ofspeech in the original text message and/or the translation correctionfrom the second user may be indicative of a language detection failure.The operations may also include rejecting the translation correctionfrom the second user when the translation correction from the seconduser is the same as the initial translation.

In another aspect, the invention relates to a computer program productstored in one or more storage media for controlling a processing mode ofa data processing apparatus. The computer program product is executableby the data processing apparatus to cause the data processing apparatusto perform operations including: providing a text message chat system toa plurality of users; receiving an original text message in a firstlanguage from a first user; generating an initial translation in asecond language of the original text message; providing the originaltext message and the initial translation to a second user; receiving atranslation correction from the second user to address an error in theinitial translation; and at least one of: (a) identifying a mostaccurate translation correction from a plurality of translationcorrections, the plurality of translation corrections including thetranslation correction from the second user; and (b) evaluating anaccuracy of the translation correction from the second user using aword-based feature, a language-based feature, and/or a word alignmentfeature.

In certain embodiments, the operations include offering an incentive(e.g., a virtual good and/or a virtual currency for use in an onlinegame) to encourage the second user to submit the translation correction.Determining the most accurate translation correction may include:

receiving at least one additional translation correction from at leastone additional user to address the error in the initial translation,wherein the at least one additional translation correction and thetranslation correction from the second user define the plurality oftranslation corrections; receiving feedback from users regarding anaccuracy of the plurality of translation corrections; and, based on thefeedback, identifying the most accurate translation correction from theplurality of translation corrections.

In some implementations, the operations also include providing a reward(e.g., a virtual good and/or a virtual currency for use in an onlinegame) to a user who submitted the most accurate translation correction.The operations may also include providing a reward (e.g., a virtual goodand/or a virtual currency for use in an online game) to a user whoprovided the feedback used to identify the most accurate translation.The word-based feature may include, for example, a word count, acharacter count, an emojis, a number, and/or a punctuation mark. Usingthe language-based feature may include identifying parts of speechpresent in the original text message and in the translation correctionfrom the second user.

In some embodiments, the operations also include: identifying a numberof verbs present in each of the original text message and thetranslation correction from the second user; and comparing the number ofverbs in the original text message with the number of verbs in thetranslation correction from the second user. An absence of a part ofspeech in the original text message and/or the translation correctionfrom the second user may be indicative of a language detection failure.The operations may also include rejecting the translation correctionfrom the second user when the translation correction from the seconduser is the same as the initial translation.

In one aspect, the invention relates to a method implemented by dataprocessing apparatus. The method includes: identifying a first languageand a second language; receiving a chatspeak audible message in thefirst language from a first person at a first chat client system whocommunicates in the first language; converting the chatspeak audiblemessage to a chatspeak text message in the first language; transformingthe chatspeak text message to a plain speak text message in the firstlanguage; translating the plain speak text message to a correspondingplain speak text message in the second language; transforming thecorresponding plain speak text message to a corresponding chatspeak textmessage in the second language; converting the corresponding chatspeaktext message to a corresponding chatspeak audible message in the secondlanguage; and sending the corresponding chatspeak audible message to asecond person at a second chat client system who communicates in thesecond language.

In certain embodiments, converting the chatspeak audible message to achatspeak text message in the first language includes providing thechatspeak audible message to a speech recognition system. Transformingthe chatspeak text message may include: identifying a chatspeak word orphrase in the chatspeak text message; and replacing the chatspeak wordor phrase with a non-chatspeak word or phrase. In some examples,converting the corresponding chatspeak text message to a correspondingchatspeak audible message in the second language includes providing thecorresponding chatspeak text message to a text-to-speech system.

In certain implementations, the speech recognition system and/or thetext-to-speech system utilize data including a plurality of accents anddialects for each of the first and second languages. The data mayinclude chatspeak and plain speak formats for each of the first andsecond languages. In various embodiments, the method includes receivingfeedback from a user regarding an accuracy of the corresponding plainspeak text message, the corresponding chatspeak text message, and/or thecorresponding chatspeak audible message. The method may also includeoffering an incentive (e.g., a virtual good and/or a virtual currency,for use in an online game) to the user for providing the feedback. Insome instances, the method includes processing the feedback to improveaccuracy of the speech recognition system and/or the text-to-speechsystem.

In another aspect, the invention relates to a system including acomputer readable medium having instructions stored thereon, and a dataprocessing apparatus. The data processing apparatus is configured toexecute the instructions to perform operations including: identifying afirst language and a second language; receiving a chatspeak audiblemessage in the first language from a first person at a first chat clientsystem who communicates in the first language; converting the chatspeakaudible message to a chatspeak text message in the first language;transforming the chatspeak text message to a plain speak text message inthe first language; translating the plain speak text message to acorresponding plain speak text message in the second language;transforming the corresponding plain speak text message to acorresponding chatspeak text message in the second language; convertingthe corresponding chatspeak text message to a corresponding chatspeakaudible message in the second language; and sending the correspondingchatspeak audible message to a second person at a second chat clientsystem who communicates in the second language.

In certain embodiments, converting the chatspeak audible message to achatspeak text message in the first language includes providing thechatspeak audible message to a speech recognition system. Transformingthe chatspeak text message may include: identifying a chatspeak word orphrase in the chatspeak text message; and replacing the chatspeak wordor phrase with a non-chatspeak word or phrase. In some examples,converting the corresponding chatspeak text message to a correspondingchatspeak audible message in the second language includes providing thecorresponding chatspeak text message to a text-to-speech system.

In certain implementations, the speech recognition system and/or thetext-to-speech system utilize data including a plurality of accents anddialects for each of the first and second languages. The data mayinclude chatspeak and plain speak formats for each of the first andsecond languages. In various embodiments, the operations includereceiving feedback from a user regarding an accuracy of thecorresponding plain speak text message, the corresponding chatspeak textmessage, and/or the corresponding chatspeak audible message. Theoperations may also include offering an incentive (e.g., a virtual goodand/or a virtual currency, for use in an online game) to the user forproviding the feedback. In some instances, the operations includeprocessing the feedback to improve accuracy of the speech recognitionsystem and/or the text-to-speech system.

In another aspect, the invention relates to a computer program productstored in one or more storage media for controlling a processing mode ofa data processing apparatus. The computer program product is executableby the data processing apparatus to cause the data processing apparatusto perform operations including: identifying a first language and asecond language; receiving a chatspeak audible message in the firstlanguage from a first person at a first chat client system whocommunicates in the first language; converting the chatspeak audiblemessage to a chatspeak text message in the first language; transformingthe chatspeak text message to a plain speak text message in the firstlanguage; translating the plain speak text message to a correspondingplain speak text message in the second language; transforming thecorresponding plain speak text message to a corresponding chatspeak textmessage in the second language; converting the corresponding chatspeaktext message to a corresponding chatspeak audible message in the secondlanguage; and sending the corresponding chatspeak audible message to asecond person at a second chat client system who communicates in thesecond language.

In certain embodiments, converting the chatspeak audible message to achatspeak text message in the first language includes providing thechatspeak audible message to a speech recognition system. Transformingthe chatspeak text message may include: identifying a chatspeak word orphrase in the chatspeak text message; and replacing the chatspeak wordor phrase with a non-chatspeak word or phrase. In some examples,converting the corresponding chatspeak text message to a correspondingchatspeak audible message in the second language includes providing thecorresponding chatspeak text message to a text-to-speech system.

In certain implementations, the speech recognition system and/or thetext-to-speech system utilize data including a plurality of accents anddialects for each of the first and second languages. The data mayinclude chatspeak and plain speak formats for each of the first andsecond languages. In various embodiments, the operations includereceiving feedback from a user regarding an accuracy of thecorresponding plain speak text message, the corresponding chatspeak textmessage, and/or the corresponding chatspeak audible message. Theoperations may also include offering an incentive (e.g., a virtual goodand/or a virtual currency, for use in an online game) to the user forproviding the feedback. In some instances, the operations includeprocessing the feedback to improve accuracy of the speech recognitionsystem and/or the text-to-speech system.

In one aspect, the invention relates to a method implemented by a dataprocessing apparatus. The method includes: selecting a mixture of oldtraining data (e.g., including one or more old text messages for whichcorrect translations to a different language are known) and new trainingdata (e.g., including one or more new text messages for which correcttranslations to the different language are not known); sending aplurality of respective requests at different times to a client deviceof a user (or to multiple client devices of multiple users), wherein therequests include (i) a respective request for the user to translate theold training data and/or the new training data and (ii) a respectiveincentive for the translation; after sending a particular request,receiving a translation from the client device for the old training dataof the particular request; comparing the received translation with thecorrect translation for the old training data; determining an accuracyof the received translation based on the comparison; and updating aconfidence score for the user based on the translation. The confidencescore represents a likelihood that the user will provide an accuratetranslation of a text message to the different language at a later time.

In certain embodiments, the user is a participant in an online game. Therespective incentive may include, for example, a virtual good and/or avirtual currency for the online game. Determining an accuracy of thetranslation received from the user may include (i) computing word errorrate (WER) and/or (ii) using bilingual evaluation understudy (BLEU). Insome instances, updating the confidence score for the user includesusing item response theory to identify a deviation from a norm in usertranslation accuracy. The method may also include revoking the user'stranslation privileges when the confidence score falls below a thresholdvalue.

In various implementations, the method includes rewarding the user withthe respective incentive when the user's translation is determined to becorrect. The method may also include detecting collusion between theuser and a second user by identifying a pre-existing relationshipbetween the user and the second user. In one example, identifying thepre-existing relationship includes analyzing a social network of atleast one of the user and the second user.

In another aspect, the invention relates to a system including acomputer readable medium having instructions stored thereon, and a dataprocessing apparatus. The data processing apparatus is configured toexecute the instructions to perform operations including: selecting amixture of old training data (e.g., including one or more old textmessages for which correct translations to a different language areknown) and new training data (e.g., including one or more new textmessages for which correct translations to the different language arenot known); sending a plurality of respective requests at differenttimes to a client device of a user (or to multiple client devices ofmultiple users), wherein the requests include (i) a respective requestfor the user to translate the old training data and/or the new trainingdata and (ii) a respective incentive for the translation; after sendinga particular request, receiving a translation from the client device forthe old training data of the particular request; comparing the receivedtranslation with the correct translation for the old training data;determining an accuracy of the received translation based on thecomparison; and updating a confidence score for the user based on thetranslation. The confidence score represents a likelihood that the userwill provide an accurate translation of a text message to the differentlanguage at a later time.

In certain embodiments, the user is a participant in an online game. Therespective incentive may include, for example, a virtual good and/or avirtual currency for the online game. Determining an accuracy of thetranslation received from the user may include (i) computing word errorrate (WER) and/or (ii) using bilingual evaluation understudy (BLEU). Insome instances, updating the confidence score for the user includesusing item response theory to identify a deviation from a norm in usertranslation accuracy. The operations may also include revoking theuser's translation privileges when the confidence score falls below athreshold value.

In various implementations, the operations include rewarding the userwith the respective incentive when the user's translation is determinedto be correct. The operations may also include detecting collusionbetween the user and a second user by identifying a pre-existingrelationship between the user and the second user. In one example,identifying the pre-existing relationship includes analyzing a socialnetwork of at least one of the user and the second user.

In another aspect, the invention relates to a computer program productstored in one or more storage media for controlling a processing mode ofa data processing apparatus. The computer program product is executableby the data processing apparatus to cause the data processing apparatusto perform operations including: selecting a mixture of old trainingdata (e.g., including one or more old text messages for which correcttranslations to a different language are known) and new training data(e.g., including one or more new text messages for which correcttranslations to the different language are not known); sending aplurality of respective requests at different times to a client deviceof a user (or to multiple client devices of multiple users), wherein therequests include (i) a respective request for the user to translate theold training data and/or the new training data and (ii) a respectiveincentive for the translation; after sending a particular request,receiving a translation from the client device for the old training dataof the particular request; comparing the received translation with thecorrect translation for the old training data; determining an accuracyof the received translation based on the comparison; and updating aconfidence score for the user based on the translation. The confidencescore represents a likelihood that the user will provide an accuratetranslation of a text message to the different language at a later time.

In certain embodiments, the user is a participant in an online game. Therespective incentive may include, for example, a virtual good and/or avirtual currency for the online game. Determining an accuracy of thetranslation received from the user may include (i) computing word errorrate (WER) and/or (ii) using bilingual evaluation understudy (BLEU). Insome instances, updating the confidence score for the user includesusing item response theory to identify a deviation from a norm in usertranslation accuracy. The operations may also include revoking theuser's translation privileges when the confidence score falls below athreshold value.

In various implementations, the operations include rewarding the userwith the respective incentive when the user's translation is determinedto be correct. The operations may also include detecting collusionbetween the user and a second user by identifying a pre-existingrelationship between the user and the second user. In one example,identifying the pre-existing relationship includes analyzing a socialnetwork of at least one of the user and the second user.

In one aspect, the invention relates to a method implemented by a dataprocessing apparatus. The method includes: obtaining a text message in afirst language, the text message comprising at least one word; providingthe text message to a machine translation system; obtaining atranslation of the text message from the machine translation system;determining that the text message and the translation both comprise theat least one word in the first language and that the at least one wordis correctly spelled; and performing one or more of the following: (a)determining a frequency with which the at least one word appears inprior text messages; (b) determining Bayesian probabilities forneighboring words that appear before and after the at least one word;and (c) performing k-means clustering to identify a cluster of wordsincluding synonyms. When the frequency exceeds a first threshold value,when the Bayesian probabilities exceed a second threshold value, and/orwhen the cluster includes the at least one word, the method includesadding the at least one word to a lexicon in a data store.

In certain embodiments, the at least one word includes or is an out ofvocabulary word. The at least one word may be or include a new chatspeakword. The method may include determining whether the lexicon in the datastore includes the at least one word. The text message may be receivedfrom a client device of a player in an online game. In various examples,the lexicon includes or consists of words in a vocabulary of the firstlanguage.

Determining Bayesian probabilities may include (i) reviewing previoususes of the at least one word in prior text messages and (ii)identifying words, if any, that appear before and after the at least oneword in the prior text messages. The Bayesian probabilities may providean indication of a likelihood that the neighboring words will appearbefore and after the at least one word in the text message.

In various implementations, identifying the cluster includes reviewingprior text messages and identifying words used in a similar context asthe at least one word in the text message. The method may also includeanalyzing syntax and semantics of the text message to determine parts ofspeech present in the text message.

In another aspect, the invention relates to a system including acomputer readable medium having instructions stored thereon, and a dataprocessing apparatus. The data processing apparatus is configured toexecute the instructions to perform operations including: obtaining atext message in a first language, the text message comprising at leastone word; providing the text message to a machine translation system;obtaining a translation of the text message from the machine translationsystem; determining that the text message and the translation bothcomprise the at least one word in the first language and that the atleast one word is correctly spelled; and performing one or more of thefollowing: (a) determining a frequency with which the at least one wordappears in prior text messages; (b) determining Bayesian probabilitiesfor neighboring words that appear before and after the at least oneword; and (c) performing k-means clustering to identify a cluster ofwords including synonyms. When the frequency exceeds a first thresholdvalue, when the Bayesian probabilities exceed a second threshold value,and/or when the cluster includes the at least one word, the methodincludes adding the at least one word to a lexicon in a data store.

In certain embodiments, the at least one word includes or is an out ofvocabulary word. The at least one word may be or include a new chatspeakword. The operations may include determining whether the lexicon in thedata store includes the at least one word. The text message may bereceived from a client device of a player in an online game. In variousexamples, the lexicon includes or consists of words in a vocabulary ofthe first language.

Determining Bayesian probabilities may include (i) reviewing previoususes of the at least one word in prior text messages and (ii)identifying words, if any, that appear before and after the at least oneword in the prior text messages. The Bayesian probabilities may providean indication of a likelihood that the neighboring words will appearbefore and after the at least one word in the text message.

In various implementations, identifying the cluster includes reviewingprior text messages and identifying words used in a similar context asthe at least one word in the text message. The operations may alsoinclude analyzing syntax and semantics of the text message to determineparts of speech present in the text message.

In another aspect, the invention relates to a computer program productstored in one or more storage media for controlling a processing mode ofa data processing apparatus. The computer program product is executableby the data processing apparatus to cause the data processing apparatusto perform operations including: obtaining a text message in a firstlanguage, the text message comprising at least one word; providing thetext message to a machine translation system; obtaining a translation ofthe text message from the machine translation system; determining thatthe text message and the translation both comprise the at least one wordin the first language and that the at least one word is correctlyspelled; and performing one or more of the following: (a) determining afrequency with which the at least one word appears in prior textmessages; (b) determining Bayesian probabilities for neighboring wordsthat appear before and after the at least one word; and (c) performingk-means clustering to identify a cluster of words including synonyms.When the frequency exceeds a first threshold value, when the Bayesianprobabilities exceed a second threshold value, and/or when the clusterincludes the at least one word, the method includes adding the at leastone word to a lexicon in a data store.

In certain embodiments, the at least one word includes or is an out ofvocabulary word. The at least one word may be or include a new chatspeakword. The operations may include determining whether the lexicon in thedata store includes the at least one word. The text message may bereceived from a client device of a player in an online game. In variousexamples, the lexicon includes or consists of words in a vocabulary ofthe first language.

Determining Bayesian probabilities may include (i) reviewing previoususes of the at least one word in prior text messages and (ii)identifying words, if any, that appear before and after the at least oneword in the prior text messages. The Bayesian probabilities may providean indication of a likelihood that the neighboring words will appearbefore and after the at least one word in the text message.

In various implementations, identifying the cluster includes reviewingprior text messages and identifying words used in a similar context asthe at least one word in the text message. The operations may alsoinclude analyzing syntax and semantics of the text message to determineparts of speech present in the text message.

In one aspect, the invention relates to a method implemented by dataprocessing apparatus. The method includes: (a) receiving a request toreview a portion of a history of text messages from a multi-user chatsession, the history comprising a plurality of text messages, each textmessage being in a respective language and having originated from arespective chat session participant; (b) performing a plurality ofparallel processes, each parallel process comprising (i) selecting adifferent respective text message from the portion of the history oftext messages, and (ii) translating the selected text message into atarget language; (c) providing translated text messages from theplurality of parallel processes to a client device of a user; (d)receiving a request to review a different portion of the history of textmessages; and (e) repeating steps (b) and (c) for the different portionof the history of text messages.

In certain embodiments, selecting the different respective text messageincludes querying a storage device for the portion of the history oftext messages. Translating the selected text message may includetransforming at least a portion of the text message from chatspeak toplain speak. In some implementations, the method includes receiving arequest from the user to stop viewing the history of text messages. Theplurality of parallel processes may include one process for eachrespective chat session participant. Alternatively or additionally, theplurality of parallel processes may include one process for eachlanguage used in the respective chat session.

In another aspect, the invention relates to a system including acomputer readable medium having instructions stored thereon, and a dataprocessing apparatus. The data processing apparatus is configured toexecute the instructions to perform operations including: (a) receivinga request to review a portion of a history of text messages from amulti-user chat session, the history comprising a plurality of textmessages, each text message being in a respective language and havingoriginated from a respective chat session participant; (b) performing aplurality of parallel processes, each parallel process comprising (i)selecting a different respective text message from the portion of thehistory of text messages, and (ii) translating the selected text messageinto a target language; (c) providing translated text messages from theplurality of parallel processes to a client device of a user; (d)receiving a request to review a different portion of the history of textmessages; and (e) repeating steps (b) and (c) for the different portionof the history of text messages.

In certain embodiments, selecting the different respective text messageincludes querying a storage device for the portion of the history oftext messages. Translating the selected text message may includetransforming at least a portion of the text message from chatspeak toplain speak. In some implementations, the operations include receiving arequest from the user to stop viewing the history of text messages. Theplurality of parallel processes may include one process for eachrespective chat session participant. Alternatively or additionally, theplurality of parallel processes may include one process for eachlanguage used in the respective chat session.

In another aspect, the invention relates to a computer program productstored in one or more storage media for controlling a processing mode ofa data processing apparatus. The computer program product is executableby the data processing apparatus to cause the data processing apparatusto perform operations including: (a) receiving a request to review aportion of a history of text messages from a multi-user chat session,the history comprising a plurality of text messages, each text messagebeing in a respective language and having originated from a respectivechat session participant; (b) performing a plurality of parallelprocesses, each parallel process comprising (i) selecting a differentrespective text message from the portion of the history of textmessages, and (ii) translating the selected text message into a targetlanguage; (c) providing translated text messages from the plurality ofparallel processes to a client device of a user; (d) receiving a requestto review a different portion of the history of text messages; and (e)repeating steps (b) and (c) for the different portion of the history oftext messages.

In certain embodiments, selecting the different respective text messageincludes querying a storage device for the portion of the history oftext messages. Translating the selected text message may includetransforming at least a portion of the text message from chatspeak toplain speak. In some implementations, the operations include receiving arequest from the user to stop viewing the history of text messages. Theplurality of parallel processes may include one process for eachrespective chat session participant. Alternatively or additionally, theplurality of parallel processes may include one process for eachlanguage used in the respective chat session.

In one aspect, the invention relates to a method implemented by dataprocessing apparatus. The method includes: providing a text message chatsystem to a plurality of users (e.g., of an online game); receiving arequest from a first user of the text message chat system to block asecond user of the text message chat system; and, following receipt ofthe request, preventing text messages from the second user from beingdisplayed for the first user.

In certain embodiments, following receipt of the request, the methodincludes blocking future invitations from the second user to the firstuser to engage in a chat session using the text message chat system. Themethod may include receiving a second request from the first user tounblock the second user. In some instances, following receipt of thesecond request, the method includes permitting text messages from thesecond user to be displayed for the first user. Following receipt of thesecond request, the method may include permitting future invitations tobe sent from the second user to the first user to engage in a chatsession using the text message chat system.

In some embodiments, the plurality of users include or define analliance in the online game. The method may include translating at leasta portion of a text message in the text message chat system from a firstlanguage to a second language. The method may also include transformingat least a portion of the text message from chat speak to plain speak.In some implementations, translating and/or transforming may include orutilize parallel processes. For example, the parallel processes mayinclude or utilize one process for each of the plurality of users of thechat system (or one process for each language used by the users).

In another aspect, the invention relates to a system including acomputer readable medium having instructions stored thereon, and a dataprocessing apparatus. The data processing apparatus is configured toexecute the instructions to perform operations including: providing atext message chat system to a plurality of users (e.g., of an onlinegame); receiving a request from a first user of the text message chatsystem to block a second user of the text message chat system; and,following receipt of the request, preventing text messages from thesecond user from being displayed for the first user.

In certain embodiments, following receipt of the request, the operationsinclude blocking future invitations from the second user to the firstuser to engage in a chat session using the text message chat system. Theoperations may include receiving a second request from the first user tounblock the second user. In some instances, following receipt of thesecond request, the operations include permitting text messages from thesecond user to be displayed for the first user. Following receipt of thesecond request, the operations may include permitting future invitationsto be sent from the second user to the first user to engage in a chatsession using the text message chat system.

In some embodiments, the plurality of users include or define analliance in the online game. The operations may include translating atleast a portion of a text message in the text message chat system from afirst language to a second language. The operations may also includetransforming at least a portion of the text message from chat speak toplain speak. In some implementations, translating and/or transformingmay include or utilize parallel processes. For example, the parallelprocesses may include or utilize one process for each of the pluralityof users of the chat system (or one process for each language used bythe users).

In another aspect, the invention relates to a computer program productstored in one or more storage media for controlling a processing mode ofa data processing apparatus. The computer program product is executableby the data processing apparatus to cause the data processing apparatusto perform operations including: providing a text message chat system toa plurality of users (e.g., of an online game); receiving a request froma first user of the text message chat system to block a second user ofthe text message chat system; and, following receipt of the request,preventing text messages from the second user from being displayed forthe first user.

In certain embodiments, following receipt of the request, the operationsinclude blocking future invitations from the second user to the firstuser to engage in a chat session using the text message chat system. Theoperations may include receiving a second request from the first user tounblock the second user. In some instances, following receipt of thesecond request, the operations include permitting text messages from thesecond user to be displayed for the first user. Following receipt of thesecond request, the operations may include permitting future invitationsto be sent from the second user to the first user to engage in a chatsession using the text message chat system.

In some embodiments, the plurality of users include or define analliance in the online game. The operations may include translating atleast a portion of a text message in the text message chat system from afirst language to a second language. The operations may also includetransforming at least a portion of the text message from chat speak toplain speak. In some implementations, translating and/or transformingmay include or utilize parallel processes. For example, the parallelprocesses may include or utilize one process for each of the pluralityof users of the chat system (or one process for each language used bythe users).

Elements of embodiments described with respect to a given aspect of theinvention may be used in various embodiments of another aspect of theinvention. For example, it is contemplated that features of dependentclaims depending from one independent claim can be used in apparatusand/or methods of any of the other independent claims.

Other features and aspects of various embodiments will become apparentfrom the following detailed description, taken in conjunction with theaccompanying drawings, which illustrate, by way of example, the featuresof such embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments are described in detail with reference to thefollowing figures. The drawings are provided for purposes ofillustration only and merely depict some embodiments. These drawingsshall not be considered limiting of the breadth, scope, or applicabilityof embodiments.

FIG. 1 is a block diagram illustrating an exemplary environmentutilizing a multi-lingual communications system, in accordance withvarious embodiments.

FIG. 2 is a block diagram illustrating an exemplary communicationtransformation and translation system, in accordance with variousembodiments.

FIG. 3 is a block diagram illustrating an exemplary transformationmodule, in accordance with various embodiments.

FIG. 4 is a block diagram illustrating an exemplary chat client system,in accordance with various embodiments.

FIG. 5 is a flowchart illustrating an exemplary method of multi-lingualcommunication, in accordance with various embodiments.

FIG. 6 is a flowchart illustrating an exemplary method of transformingcommunications, in accordance with various embodiments.

FIG. 7 is a diagram illustrating an exemplary multi-lingual chat sessionbetween chat client systems, in accordance with various embodiments.

FIG. 8 is a flowchart illustrating operation of an exemplarymulti-lingual communication method, in accordance with variousembodiments.

FIG. 9 is a flowchart illustrating operation of an exemplarymulti-lingual communication method, in accordance with variousembodiments.

FIG. 10 is a flowchart illustrating operation of an exemplarymulti-lingual communication method, in accordance with variousembodiments.

FIG. 11 is a flowchart illustrating operation of an exemplarymulti-lingual communication method, in accordance with variousembodiments.

FIG. 12 is a block diagram illustrating an exemplary digital device thatcan be utilized, in accordance with various embodiments.

FIG. 13 is a block diagram illustrating an example user feedback system,in accordance with various embodiments.

FIG. 14 is a block diagram illustrating an example user feedback clientsystem, in accordance with various embodiments.

FIG. 15 is a flowchart illustrating an example method for user feedback,in accordance with various embodiments.

FIG. 16 is a block diagram illustrating an example data flow for a userfeedback system, in accordance with various embodiments.

FIG. 17 depicts screenshots illustrating an example of receiving userfeedback for a word, in accordance with various embodiments.

FIG. 18 depicts screenshots illustrating an example of skipping userfeedback, in accordance with various embodiments.

FIG. 19 depicts screenshots illustrating an example of receiving userfeedback for a phrase, in accordance with various embodiments.

FIG. 20 depicts screenshots illustrating an example of receiving userfeedback through a listing of select-form responses, in accordance withvarious embodiments.

FIG. 21 depicts screenshots illustrating an example of creating alisting of select-form responses, in accordance with variousembodiments.

FIG. 22 depicts screenshots illustrating example incentivenotifications, in accordance with various embodiments.

FIG. 23 depicts screenshots illustrating an example of when atranslation has failed between client chat systems, in accordance withvarious embodiments.

FIGS. 24 and 25 depict screenshots illustrating example listings ofwords or phrases available for user feedback, in accordance with variousembodiments.

FIG. 26 depicts a screenshot illustrating an example of defining a word,in accordance with various embodiments.

FIG. 27 depicts a screenshot illustrating an example listing ofselect-form responses, in accordance with various embodiments.

FIG. 28 depicts a screenshot illustrating an example listing of statusesfor responses submitted, in accordance with various embodiments.

FIG. 29 depicts a screenshot illustrating an example incentivenotification, in accordance with various embodiments.

FIG. 30 is a flowchart for a method of detecting undecipherable phrasesin a language corpus, in accordance with various embodiments.

FIG. 31A is a schematic diagram of a fraud detection module, inaccordance with various embodiments.

FIG. 31B is a flowchart of a method of detecting fraud in incentivizedtranslations, in accordance with various embodiments.

FIG. 32 is a schematic diagram of a system for performing speech-to-texttranscription and translation in a group chat environment, in accordancewith various embodiments.

FIG. 33A is a schematic diagram of a chat history module fortransforming and/or translating chat histories, in accordance withvarious embodiments.

FIG. 33B is a flowchart of a method of transforming and/or translatingchat histories, in accordance with various embodiments.

FIG. 34A includes screenshots of a user interface for blocking one ormore users of a chat session, in accordance with various embodiments.

FIG. 34B includes screenshots of a user interface for unblocking one ormore users of a chat session, in accordance with various embodiments.

FIG. 35 includes a flowchart of a method of blocking one or more usersof a chat session, in accordance with various embodiments.

FIGS. 36A-36F include screenshots of a user interface that allows a userto correct a language translation of an original message, in accordancewith various embodiments.

FIGS. 37A and 37B include screenshots of a user interface that allowsusers to judge translation corrections submitted by other users, inexchange for a possible a reward, in accordance with variousembodiments.

FIG. 38 is a schematic diagram of a translation accuracy module forevaluating the accuracy of translations, in accordance with variousembodiments.

FIG. 39 is a flowchart of a method of evaluating the accuracy oftranslations, in accordance with various embodiments.

DETAILED DESCRIPTION

Various embodiments described herein relate to and facilitatemulti-lingual communications. The systems and methods of someembodiments may enable multi-lingual communications through differentmodes of communications including, for example, Internet-based chat(e.g., Apple® iMessage, Windows® Live Messenger, etc.), e-mail (e.g.,embedded forum messaging, Yahoo® mail, RFC 5322, etc.), text-basedmobile phone communications (e.g., SMS messages or MMS messages),postings to online forums (e.g., postings to a web-based hobby forum),postings to online social media services (e.g., Twitter®, Facebook®,etc.), and the like. Certain embodiments may also be used to translatetranscripts of communications or conversations that took place in thepast (e.g., deposition transcripts or chat history). Various embodimentsmay implement communications systems and methods that translate textbetween two or more languages (e.g., spoken), whilehandling/accommodating for one or more of the following in the text:specialized/domain-related jargon (e.g., chatspeak), abbreviations,acronyms, proper nouns, common nouns, diminutives, colloquial words orphrases, and profane words or phrases. For example, some systems andmethods described herein may be utilized in connection with a chatsystem, such as those used in massive-multiplayer online (MMO) games,which tend to have users that chat in different foreign languages.Through certain embodiments, the chat dialogue between two or more userscan be transparently translated and presented to each user in theirrespective native language or language of choice. Additionally, throughthe use of a multi-tiered/multi-module transformation process, certainembodiments may facilitate faster translation of communication betweentwo or more users (e.g., in their respective native languages) thanotherwise possible by traditional translation systems alone (e.g.,translation in a matter of microseconds).

According to some embodiments, a system or method may performtranslation from chatspeak in a first language, such as English, tochatspeak in a second language, such as French. In another example, asystem or method may perform transformation from chatspeak in the firstlanguage (e.g., English) to formal speak in the first language (e.g.,English), before attempting translation to the second language (e.g.,French). Some embodiments may achieve such text translations by firstquerying a data store (e.g., translations cache), which may containtranslations manually entered by a human operator or translations basedon previously performed by a translation system (e.g., historicaltranslations performed by an embodiment). Embodiments may attempt totransform one or more portions of the text (e.g., process one or more ofthe following within the text: chatspeak, acronyms, abbreviations,proper nouns, common nouns, colloquialisms, and profanity) to make itmore suitable for accurate text translation. For example, certainembodiments may transform a given text to account for (current or past)idiomatic language use across different languages. Embodiments mayreattempt querying the data store after transformation of the portionsof the text. If this translation lookup reattempt fails, embodiments mayattempt to translate the text (which may have been transformed) using amachine translation service (e.g., third-party, cloud-based translationservice, such as Google® translate).

Embodiments may attempt to transform a translated piece of formal textto chatspeak in the new language (e.g., transform French formal speak toFrench chatspeak) to further refine the translation of the texteventually produced. Accordingly, certain embodiments facilitate chattranslation between chatspeak in a first language (e.g., English) tochatspeak in a second language (e.g., Russian, French, Spanish, Chinese,Hindi, etc.).

Some embodiments may help reduce or avoid the need for using machinetranslations (thereby reducing time, cost, and other overhead associatedwith machine translations), and may facilitate accurate translations oftext having minimal context or comprising short sentence structure.Where the machine translation is facilitated by a third-party service orover a secure network connection (e.g., Secure-Socket Layer [SSL]connection), the cost or overhead avoided by certain embodiments may besignificant.

As understood herein, “transformation” means manipulating a first textsegment, in a first language, to form a second text segment in the firstlanguage. The resulting second text segment may also be referred toherein as the “transformed text.” “Translation” will be understood tomean converting a text segment in a first language to a correspondingtext segment in a second language.

As also understood herein, a “transformed translation” means translationof a text segment (from a first language to a second language) that hasalready been transformed in accordance with embodiments described herein(e.g., transformed from chatspeak text in a first language to formaltext in the first language). An “untransformed translation” will beunderstood to mean a translation of a text segment (from a firstlanguage to a second language) before the text segment has beentransformed in accordance with embodiments described herein.

Various embodiments may implement different transformation/translationstrategies, with certain strategies being well suited for particulartranslation applications. For example, for particular chat systemapplications, the transformation strategy implemented may compriseapplying the following set of transformation-related modules in theorder listed: chatspeak module, acronym module, proper noun module,common noun module, colloquialism module, spelling check module,abbreviation module, and profanity module. Generally, thetransformation/translation strategy employed determines whichtransformation operations are performed, when the transformationoperations are performed in the overall translation process (e.g.,transformation performed before or after machine translation), or inwhat order the transformation operations are performed (e.g., precedenceor priority of transformation operations). Thetransformation/translation strategy may also determine what translationsare pre-populated into the data store (e.g., translations can be storedin a translation “cache” to speed up the overall process) and whentranslation caches are utilized in the overall translation process. Forcertain embodiments, the transformation/translation strategy employedmay be dynamically determined based on the conditions of the environmentin which the embodiments are used. For example, where a chat system isexperiencing a heavier load of users than usual, thetransformation/translation strategy may switch to one that lessens theprocessing burden of the chat system (e.g., relies more on machinetranslations rather than on the data store).

FIG. 1 is a block diagram illustrating an exemplary environment 100utilizing a multi-lingual system in accordance with various embodiments.As shown in FIG. 1, the exemplary environment 100 comprises clients102-1 through 102-N (hereafter, collectively referred to as “clients102” or “client 102”), a chat server 108, and a translation server 110,each of which may be communicatively coupled with each other through acomputer network 106. In accordance with some embodiments, the computernetwork 106 may be implemented or facilitated using one or more local orwide-area communications networks, such as the Internet, WiFi networks,WiMax networks, private networks, public networks, and the like.Depending on the embodiment, some or all of the communicationconnections with the computer network 106 may utilize encryption (e.g.,Secure Sockets Layer [SSL]) to secure information being transferredbetween the various entities shown in the exemplary environment 100.

Each of the clients 102, the chat server 108, and the translation server110 may be implemented using one or more digital devices, which may besimilar to the digital devices discussed later with respect to FIG. 12.For instance, the client 102-1 may be any form of computing devicecapable of receiving user input (e.g., configured for user interaction),capable of providing a client user interface that facilitatescommunications with one or more other clients (e.g., any of clients102-2 through 102-N), and capable of communicating with the chat server108 through the computer network 106. Such computing devices may includea mobile phone, a tablet computing device, a laptop, a desktop computer,personal digital assistant, a portable gaming unit, a wired gaming unit,a thin client, a set-top box, a portable multi-media player, or anyother type of network accessible user device known to those of skill inthe art. Further, one or more of the chat server 108 and the translationserver 110 may comprise of one or more servers, which may be operatingon or implemented using one or more cloud-based services (e.g.,System-as-a-Service [SaaS], Platform-as-a-Service [PaaS], orInfrastructure-as-a-Service [IaaS]).

The clients 102 may be configured to communicatively connect with thechat server 108, which provides or otherwise facilitates chat sessionsbetween the clients 102. Each of the clients 102-1 through 102-N maycomprise a chat client system (104-1 through 104-N, respectively) thatenables a user at each of the clients 102 to access to the chat sessionthrough the chat server 108. Additionally, depending on the embodiment,each of the chat client systems 104-1 through 104-N (hereafter,collectively referred to as “chat client systems 104” or “chat clientsystem 104”) may be implemented as a standalone chat application, as achat feature embedded in non-chat application (e.g., video game), orthrough a chat service accessible at the client through a web browser.Those skilled in the art will appreciate that for some embodiments thechat client systems 104 may be non-heterogeneous with respect to oneanother and still be capable of establishing a chat session betweenthem. The chat client systems 104 may be capable of receiving chat input(e.g., a chat message) from their respective users in a language (andcorresponding character set) selected by the user (e.g., based on usersettings or preferences), and transmitting the chat input to the chatserver 108 to be relayed to another user (e.g., another user at anotherchat client system). The chat client systems 104 may also be capable ofreceiving chat output (e.g., chat session dialogue) from the chat server108 (e.g., from another user at another chat client system), anddisplaying the received chat output in a language (and correspondingcharacter set) selected by the user (e.g., based on user settings orpreferences).

Through the use of some embodiments, the translation of the chatdialogue may be transparent to the users as dialogue is passed betweenthe chat client systems 104. Accordingly, for some embodiments, all chatdialogue presented at a given chat client system 104 may be in alanguage native to (or selected by) the user at that given chat clientsystem 104, irrespective of what language is being by users, at otherchat client systems 104 that are contributing to the same chat dialogue.For example, where the user at the chat client system 104-1 and the userat the chat client system 104-2 are contributing to the same chatdialogue (i.e., involved in the same chat session), the user at the chatclient system 104-1 may have chosen to enter and receive chat dialoguein English while the user at the chat client system 104-2 may havechosen to enter and receive chat dialogue in Russian. Though the usersat the client systems 104-1 and 104-2 will see the same chat content,the chat dialogue will be presented in their respectively chosenlanguages.

As shown, the chat server 108 may comprise a chat host system 112configured to established and/or facilitate chat sessions between thechat client systems 104, and a communication transformation andtranslation (CTT) system 114 configured to perform transformation and/ortranslation operations in accordance with the various systems andmethods described herein. For some embodiments, the chat client systems104 may establish a chat session with each other through the chat hostsystem 112, and the chat host system 104 may utilize the features of theCTT system 114 in facilitating the transparent translation of chatdialogue between the chat client systems 104. Those skilled in the artwill appreciate that for some embodiments, the chat host system 112 andthe CTT system 114 may be part of separate servers, and that the entityoperating the chat host system 112 may be different from the entityoperating the CTT system 114. For instance, the chat host system 112 maybe a third-party chat host system that utilizes the services of the CTTsystem 114.

As also shown, the translation server 110 may comprise a translationmodule 116 configured to receive and service requests for machine texttranslation. In accordance with some embodiments, the CTT system 114 mayutilize the operations/services of the translation module 116 inperforming machine translations of texts. The CTT system 114 may use ofone or more translation application programming interfaces (APIs) toobtain access to the services provided by the translation module 116.Depending on the embodiment, the translation module 116 (and the server110 on which it resides) may be operated by a third-party, such asGoogle®, which may offer the machine translation services free of chargeor for a fee. Though the translation module 116 is shown to be acomponent operating on a server separate from the CTT system 114, thoseskilled in the art will appreciate that, for some embodiments, thetranslation module 116 may operating on the same server as the CTTsystem 114 and/or may be an integrated component of the CTT system 114.

FIG. 2 is a block diagram illustrating an exemplary communicationtransformation and translation system 114 in accordance with variousembodiments. As shown, the CTT system 114 may comprise a communicationtransformation and translation (CTT) control module 202, a communicationtransformation and translation (CTT) communications module 204, alanguage module 206, a transformation module 208, a translation datastore 210, and a translation application programming interface (API)module 212. The CTT control module 202 may be configured to controland/or orchestrate performance of various operations within the CTTsystem 114 as the CTT system 114 performs transformation or translationoperations in accordance with some systems and methods described herein.For some embodiments, the CTT control module 202 may control theoperation of other components of the CTT system 114, such as the CTTcommunications module 204, the language module 206, the transformationmodule 208, the translation data stores 210, and the translation APImodule 212.

The CTT communications module 204 may be configured to facilitatecommunications between the CTT system 114 and systems and componentsexternal to the CTT system 114, such as the chat server 108 and/or thetranslation server 110. Accordingly, through the CTT communicationsmodule 204, the CTT system 114 may receive the chat dialogue (comprisingone or more chat messages) to be transformed or translated by the CTTsystem 114, and may output the translated chat dialogue that resultsfrom the CTT system 114.

The language module 206 may be configured to identify the one or morelanguages used in connection with chat dialogue received by the CTTsystem 114. For some embodiments, the language module 206 may identifythe language through analysis of the content of the chat dialoguereceived, and/or obtaining language preference/settings information fromthe respective chat client systems (e.g., chat client systems 104)involved with the chat dialogue received.

The transformation module 208 may be configured to performtransformation operations on chat dialogue (comprising one or more chatmessages), received by the CTT system 114, in accordance with somesystems and methods described herein. In accordance with someembodiments, the transformation operations performed by thetransformation module 208 may include, without limitation, thoserelating to chatspeak, acronyms, abbreviations, proper nouns, commonnouns, colloquialisms, and profanity. Additional details of thetransformation module 208 are discussed in FIG. 3.

The translation data store 210 may be configured to store andsubsequently provide previously translated text to the CTT system 114 asthe CTT system 114 performs transformed translations and untransformedtranslations in accordance with the some system and methods describedherein. As described herein, the translation data store 210 may operateas a cache for translations previously performed by the CTT system 114,and/or may store translations manually entered and stored by a humanoperator (e.g., by way of a translation training system). For someembodiments, the translation data store 210 may be populated withtranslations that would speed up the performance of the CTT system 114with respect to certain chat contexts. For example, where the CTT system114 is utilized in conjunction with a chat system associated with an MMOgame, the translation data store 210 may be populated (e.g., by theoperator of the CTT system 114) with (transformed and untransformed)translations relating specifically to the MMO game. For certainembodiments, the multi-tiered/multi-module approach of transforming textused by the transformation module 208 is particularly well suited forhandling chat text in MMO games, which by nature tends to be complex.

Depending on the embodiment, the data store 210 may store eitheruntransformed translations (e.g., <English Formal>‘you’→<FrenchFormal>‘vous’), transformed translations (e.g., <EnglishChatspeak>‘u’→<French Formal>‘vous’), or both. For some embodiments, thetranslation data store 210 may store translations such thatcorresponding chat messages may be identified using hash values/tags.For instance, to store a Spanish translation for an original message inEnglish, the Spanish translation may be stored based on a hash value ofthe English message, thereby enabling the Spanish translation to belater located and obtained using the hash value of the English message.Those skilled in the art will appreciate that for some embodiments, thetranslation data store 210 may comprise a separate data store fortranslations between two specific languages. Accordingly, when a chatmessage is being transformed/translated between English and French, acorresponding data English-French data store may be utilized foroperations relating to the translation data store 210.

The translation API module 212 may be configured to provide the CTTsystem 114 with access to machine translation services provided externalto the CTT system 114 (e.g., by the translation module 116 of thetranslation server 110). As described herein, the translation API module212 may be utilized by the CTT system 114 when a translation is notlocated in the translation data store 210.

FIG. 3 is a block diagram illustrating an exemplary transformationmodule 208 in accordance with various embodiments. As shown, thetransformation module 208 may comprise a chatspeak module 302, anacronym module 304, a proper noun module 306, a common noun module 308,a colloquialism module 310, a spelling check module 312, an abbreviationmodule 314, and/or a profanity module 316. According to someembodiments, during operation the transformation module 208 may processa chat message in whole or in parts (e.g., breaks the message intotokens or logical portions and then processes those tokens/portions). Insome embodiments, various modules of the transformation module 208 maybe called in parallel.

The chatspeak module 302 may be configured to identify one or more wordsor phrases in a chat message that are associated with chat jargon (i.e.,chatspeak), and may be further configured to suggest replacement (e.g.,corresponding formal/i.e., non-chatspeak) words or phrases for theidentified words or phrases. In some embodiments, the chatspeak module302 may flag an identified chatspeak word or phrase to be skipped orotherwise ignored during a subsequent machine translation (e.g., by thetranslation module 116). Additionally, in some embodiments, anidentified chatspeak word or phrase may be flagged for later review anddisposition by a human operator (e.g., an administrator of the CTTsystem 114). In order to identify a chatspeak word or phrase and/or itscorresponding (formal) word or phrase, some embodiments may utilize adataset (e.g., stored on a data store) comprising chatspeak words orphrases and/or mappings between chatspeak words or phrases and theircorresponding words and phrases. The dataset may be constructed by wayof training or a learning system, may be proprietary (e.g., manuallycollected “in-house” by an administrator of the CTT system 114), may becommercially acquired, or may be derived from a publicly availableInternet knowledgebase. For example, the chatspeak module 302 may employstatistical machine translation in its functionality. For someembodiments, the statistical machine translation employed may be trainedusing parallel texts and/or using phrase-level pairs extracted fromtransformations that preserve contextual information and/or add grammarto an otherwise ungrammatical sentence. The result from the chatspeakmodule 302 may comprise a chatspeak word or phrase flagged by thechatspeak module 302 to be ignored, a suggested replacement, or anon-chatspeak word or phrase inserted into the message by the chatspeakmodule 302 (e.g., in place of the identified chatspeak word or phrase).Depending on the embodiment, the message that results from the chatspeakmodule 302 may be provided to another transformation module (in thetransformation module 208) for further processing or the suggestedreplacement may be provided to the CTT control module 202 to determineif the message transformed by the chatspeak module 302 is in the datastore 210.

The acronym module 304 may be configured to identify one or moreacronyms in a chat message, and may be further configured to suggestreplacement words or phrases corresponding to (e.g., represented by) theacronyms. In some embodiments, the acronym module 304 may flag anidentified acronym to be skipped or otherwise ignored during asubsequent machine translation (e.g., by the translation module 116).Additionally, in some embodiments, an identified acronym may be flaggedfor later review and disposition by a human operator (e.g., anadministrator of the CTT system 114). In order to identify an acronymand/or its corresponding word or phrase, some embodiments may utilize adataset (e.g., stored on a data store) comprising acronyms and/ormappings between acronyms and their corresponding words and phrases. Thedataset may be constructed by way of training or a learning system, maybe proprietary (e.g., manually collected “in-house” by an administratorof the CTT system 114), may be commercially acquired, or may be derivedfrom a publicly available Internet knowledgebase. The result from theacronym module 304 may comprise an acronym flagged by the acronym module304 to be ignored, a suggested replacement, or a word or phrase insertedinto the message by the acronym module 304 (e.g., in place of theidentified acronym). Depending on the embodiment, the message thatresults from the acronym module 304 may be provided to anothertransformation module (in the transformation module 208) for furtherprocessing or the suggested replacement may be provided to the CTTcontrol module 202 to determine if the message transformed by theacronym module 304 is in the data store 210.

The proper noun module 306 may be configured to identify one or moreproper nouns in a chat message, and may be further configured to suggestreplacement words or phrases corresponding to (e.g., represented by) theproper nouns. In some embodiments, the proper noun module 306 may flagan identified proper noun to be skipped or otherwise ignored during asubsequent machine translation (e.g., by the translation module 116).Additionally, in some embodiments, an identified proper noun may beflagged for later review and disposition by a human operator (e.g., anadministrator of the CTT system 114). In order to identify a proper nounand/or its corresponding word or phrase, some embodiments may utilize adataset (e.g., stored on a data store) comprising proper nouns (e.g.,well-known proper nouns such as Disneyland®, or common names forindividuals) and/or mappings between proper nouns and theircorresponding words and phrases. The dataset may be constructed by wayof training or a learning system, may be proprietary (e.g., manuallycollected “in-house” by an administrator of the CTT system 114), may becommercially acquired, or may be derived from a publicly availableInternet knowledgebase. The result from the proper noun module 306 maycomprise a proper noun flagged by the proper noun module 306 to beignored, a suggested replacement, or a word or phrase inserted into themessage by the proper noun module 306 (e.g., in place of the identifiedproper noun). Depending on the embodiment, the message that results fromthe proper noun module 306 may be provided to another transformationmodule (in the transformation module 208) for further processing or thesuggested replacement may be provided to the CTT control module 202 todetermine if the message transformed by the proper noun module 306 is inthe data store 210.

The common noun module 308 may be configured to identify one or morecommon nouns in a chat message, and may be further configured to suggestreplacement words or phrases corresponding to (e.g., represented by) thecommon nouns. In some embodiments, the common noun module 308 may flagan identified common noun to be skipped or otherwise ignored during asubsequent machine translation (e.g., by the translation module 116).Additionally, in some embodiments, an identified common noun may beflagged for later review and disposition by a human operator (e.g., anadministrator of the CTT system 114). In order to identify a common nounand/or its corresponding word or phrase, some embodiments may utilize adataset (e.g., stored on a data store) comprising common nouns and/ormappings between common nouns and their corresponding words and phrases.The dataset may be constructed by way of training or a learning system,may be proprietary (e.g., manually collected “in-house” by anadministrator of the CTT system 114), may be commercially acquired, ormay be derived from a publicly available Internet knowledgebase. Theresult from the common noun module 308 may comprise a common nounflagged by the common noun module 308 to be ignored, a suggestedreplacement, or a word or phrase inserted into the message by the commonnoun module 308 (e.g., in place of the identified common noun).Depending on the embodiment, the message that results from the commonnoun module 308 may be provided to another transformation module (in thetransformation module 208) for further processing or the suggestedreplacement may be provided to the CTT control module 202 to determineif the message transformed by the common noun module 308 is in the datastore 210.

The colloquialism module 310 may be configured to identify one or morecolloquial words or phrases in a chat message, and may be furtherconfigured to suggest replacement (e.g., corresponding formal/i.e.,non-colloquial) words or phrases for the identified words or phrases. Insome embodiments, the colloquialism module 310 may flag an identifiedcolloquial word or phrase to be skipped or otherwise ignored during asubsequent machine translation (e.g., by the translation module 116).Additionally, in some embodiments, an identified colloquial word orphrase may be flagged for later review and disposition by a humanoperator (e.g., an administrator of the CTT system 114). In order toidentify a colloquial word or phrase and/or its corresponding (formal)word or phrase, some embodiments may utilize a dataset (e.g., stored ona data store) comprising colloquial words or phrases and/or mappingsbetween colloquial words or phrases and their corresponding words andphrases. The dataset may be constructed by way of training or a learningsystem, may be proprietary (e.g., manually collected “in-house” by anadministrator of the CTT system 114), may be commercially acquired, ormay be derived from a publicly available Internet knowledgebase. Theresult from the colloquialism module 310 may comprise a colloquial wordor phrase flagged by the colloquialism module 310 to be ignored, asuggested replacement, or a non-colloquial word or phrase inserted intothe message by the colloquialism module 310 (e.g., in place of theidentified colloquial word or phrase). Depending on the embodiment, themessage that results from the colloquialism module 310 may be providedto another transformation module (in the transformation module 208) forfurther processing or the suggested replacement may be provided to theCTT control module 202 to determine if the message transformed by thecolloquialism module 310 is in the data store 210.

The spelling check module 312 may be configured to identify one or moremisspelled words or phrases in a chat message, and may be furtherconfigured to suggest replacement (e.g. corrected) words or phrases forthe identified words or phrases. For example, the spelling check module312 may be configured to automatically correct the words or phrases withthe suggested replacement words or phrases. In some embodiments, thespelling check module 312 may flag an identified misspelled word orphrase to be skipped or otherwise ignored during a subsequent machinetranslation (e.g., by the translation module 116). Additionally, in someembodiments, an identified misspelled word or phrase may be flagged forlater review and disposition by a human operator (e.g., an administratorof the CTT system 114). In order to identify a misspelled word or phraseand/or its corresponding (corrected) word or phrase, some embodimentsmay utilize a dataset (e.g., stored on a data store) comprisingmisspelled words or phrases and/or mappings between misspelled words orphrases and their corresponding words and phrases. The dataset may beconstructed by way of training or a learning system, may be proprietary(e.g., manually collected “in-house” by an administrator of the CTTsystem 114), may be commercially acquired, or may be derived from apublicly available Internet knowledgebase. The result from the spellingcheck module 312 may comprise a misspelled word or phrase flagged by thespelling check module 312 to be ignored, a suggested replacement, or acorrected word or phrase inserted into the message by the spelling checkmodule 312 (e.g., in place of the misspelled word or phrase). Dependingon the embodiment, the message that results from the spelling checkmodule 312 may be provided to another transformation module (in thetransformation module 208) for further processing or the suggestedreplacement may be provided to the CTT control module 202 to determineif the message transformed by the spelling check module 312 is in thedata store 210.

The abbreviation module 314 may be configured to identify one or moreabbreviations in a chat message, and may be further configured tosuggest replacement words or phrases corresponding to (e.g., representedby) the abbreviations. In some embodiments, the abbreviation module 314may flag an identified abbreviation to be skipped or otherwise ignoredduring a subsequent machine translation (e.g., by the translation module116). Additionally, in some embodiments, an identified abbreviation maybe flagged for later review and disposition by a human operator (e.g.,an administrator of the CTT system 114). In order to identify anabbreviation and/or its corresponding word or phrase, some embodimentsmay utilize a dataset (e.g., stored on a data store) comprisingabbreviations and/or mappings between abbreviations and theircorresponding words and phrases. The dataset may be constructed by wayof training or a learning system, may be proprietary (e.g., manuallycollected “in-house” by an administrator of the CTT system 114), may becommercially acquired, or may be derived from a publicly availableInternet knowledgebase. The result from the abbreviation module 314 maycomprise an abbreviation flagged by the abbreviation module 314 to beignored, a suggested replacement, or a word or phrase inserted into themessage by the abbreviation module 314 (e.g., in place of the identifiedabbreviation). Depending on the embodiment, the message that resultsfrom the abbreviation module 314 may be provided to anothertransformation module (in the transformation module 208) for furtherprocessing or the suggested replacement may be provided to the CTTcontrol module 202 to determine if the message transformed by theabbreviation module 314 is in the data store 210.

The profanity module 316 may be configured to identify one or moreprofane words or phrases (hereafter, referred to as a “profanity”) in achat message, and may be further configured to suggest replacement wordsor phrases (e.g., suitable substitute) corresponding to the profanity(e.g., a toned down euphemism). In some embodiments, the profanitymodule 316 may flag identified profanity to be skipped or otherwiseignored during a subsequent machine translation (e.g., by thetranslation module 116). Additionally, in some embodiments, identifiedprofanity may be flagged for later review and disposition by a humanoperator (e.g., an administrator of the CTT system 114). In order toidentify profanity and/or its corresponding word or phrase, someembodiments may utilize a dataset (e.g., stored on a data store)comprising profanity and/or mappings between abbreviations and theircorresponding words and phrases. The dataset may be constructed by wayof training or a learning system, may be proprietary (e.g., manuallycollected “in-house” by an administrator of the CTT system 114), may becommercially acquired, or may be derived from a publicly availableInternet knowledgebase. The result from the profanity module 316 maycomprise profanity flagged by the profanity module 316 to be ignored, asuggested replacement, or a word or phrase inserted into the message bythe profanity module 316 (e.g., in place of the identified profanity).Depending on the embodiment, the message that results from the profanitymodule 316 may be provided to another transformation module (in thetransformation module 208) for further processing or the suggestedreplacement may be provided to the CTT control module 202 to determineif the message transformed by the profanity module 316 is in the datastore 210.

For some embodiments, one or more various modules of the transformationmodule 208 may flag one or more portions of the chat message byinserting a predetermined character before and/or after the portionbeing flagged. For instance, where the chatspeak module 302 flags theword “LOL” in a portion of the chat message, the chatspeak module 302may insert an predetermined character (“_”) before and/or after the word(e.g., “_LOL_”) to indicate that the flagged portion should be ignoredby the translation module 116.

For some embodiments, the transformation module 208 may perform two ormore transformation operations on the initial message in parallel, andin response, each of the two or more transformation operations mayreturn a separate response, from which the transformation module 208 maythen select one transformed message for further processing (e.g., to beused in operation 514). Depending on the embodiment, each response maycomprise a flagged text portion, a suggested replacement, or a word orphrase inserted into the initial message. Thereafter, the transformedmessage selected may be according to a priority of selection, which candetermine which transformed message is selected for further processingand according to what precedent. In some embodiments, the priorityselection may be according to which transformation operation is mostlikely to generate a transformed message suitable for a subsequentlookup in the translation data store 210) or for subsequent machinetranslation. Additionally, in some embodiments, the priority ofselection may be according to which transformation operation generatesthe most formal transformed message. The priority of selection maydepend on the transformation/translation strategy selected by theembodiment.

Table 1 following provides examples of how the transformation module 208may process a portion of a chat message in accordance with variousembodiments. As shown, the transformation module 208 may process a chatmessage based on tokens or proximal tokens, and may cease processing ona particular token once a transformation is performed.

TABLE 1 Examples of chat message processing. Token(s) from a ChatMessage Transformation Processing Token = ‘USA’ Chatspeak Module (‘USA’)→ Acronym Module (‘USA’) → Flag for non-translation. Token = ‘brb’Chatspeak Module (‘brb’) → Acronym Module (‘brb’) → Proper Noun Module(‘brb’) → Common Noun Module (‘brb’) → Colloquialism Module (‘brb’) →Spelling Check Module (‘brb’) → Abbreviation Module (‘brb’) → Transformto ‘be right back’ Token = ‘9’ Chatspeak Module (‘9’) → Transform to‘parents watching over shoulder’ Token = ‘99’ Chatspeak Module (‘99’) →Transform to ‘parents stopped watching over shoulder’ Proximal tokens =Chatspeak Module (‘go gabe’) → Acronym ‘go gabe’ Module (‘go gabe’) →Proper Noun Module (‘going’) → Common Noun Module (‘go gabe’) → Flag forlikely being a common noun. String = ‘Your going Spelling Check Module(‘Your’) → Correct to attack him?’ with ‘You're’ based on proximal token‘going’ Token#1 = ‘Your’ (i.e., using proximal context for spell Token#2= ‘going’ checking). Token#3 = ‘to’ Chatspeak Module (‘going’) → AcronymToken#4 = ‘attack’ Module (‘going’) → Proper Noun Module Token#5 = ‘him’(‘going’) → Common Noun Module (‘going’) → Colloquialism Module(‘going’) → Spelling Check Module (‘going’) → Abbreviation Module(‘going’) → Profanity Module (‘going’) → No transform. Chatspeak Module(‘to’) → Acronym Module (‘to’) → Proper Noun Module (‘to’) → Common NounModule (‘to’) → Colloquialism Module (‘to’) → Spelling Check Module(‘to’) → Abbreviation Module (‘to’) → Profanity Module (‘to’) → Notransform. Chatspeak Module (‘attack’) → Acronym Module (‘attack’) →Proper Noun Module (‘attack’) → Common Noun Module (‘attack’) →Colloquialism Module (‘attack’) → Spelling Check Module (‘attack’) →Abbreviation Module (‘attack’) → Profanity Module (‘attack’) → Notransform. Chatspeak Module (‘him’) → Acronym Module (‘him’) → ProperNoun Module (‘him’) → Common Noun Module (‘him’) → Colloquialism Module(‘him’) → Spelling Check Module (‘him’) → Abbreviation Module (‘him’) →Profanity Module (‘him’) → No transform. String = ‘Sup bro, ChatspeakModule (‘Sup’) Replace with sup yall?’ “How is it going.” Token#1 =‘Sup’ Chatspeak Module (‘bro’) → Acronym Token#2 = ‘bro’ Module (‘bro’)→ Proper Noun Module Token#3 = ‘sup’ (‘bro’) → Common Noun Module(‘bro’) → Token#4 = ‘yall’ Colloquialism Module (‘bro’) → Spelling CheckModule (‘bro’) → Abbreviation Module (‘bro’) → Replace with “brother”Chatspeak Module (‘sup’) Replace with “how is it going.” ChatspeakModule (‘yall’) → Replace with “you all.”

FIG. 4 is a block diagram illustrating an exemplary chat client system104 in accordance with various embodiments. As shown, the chat clientsystem 104 may comprise a chat client controller 402, a chat clientcommunications module 404, and a chat client graphical user interface(GUI) module 406. The chat client control module 402 may be configuredto control and/or orchestrate performance of various operations withinthe chat client system 104 as the chat client system 104 performs chatrelated operations (e.g., communications chat dialogue with the chatserver 108). For some embodiments, the chat client control module 402may control the operation of other components of the chat client system104 including, for example, such as the chat client communicationsmodule 404, and the chat client GUI module 406.

The chat client communications module 404 may be configured tofacilitate communications between the chat client system 104 and systemsand components external to the chat client system 104, such as the chatserver 108. Accordingly, through the chat client module 404, the chatclient system 104 may receive from the chat server 108 the chat dialogueto be presented at the chat client system 104 (e.g., via the chat clientGUI module 406), and may send to the chat server the chat dialoguereceived from the user at the chat client system 104 (e.g., via the chatclient GUI module 406).

The chat client GUI module 406 may be configured to provide a user atthe chat client system 104 with graphical input/output access to chatsessions with other chat client systems. Accordingly, for someembodiments the chat client GUI module 406 may present a user at theclient 102 with a client GUI adapted for receiving user interactionsthrough the client 102. For some embodiments, the chat client GUI module406 may be configured to present the user with chat dialogue (e.g., asthey are received from the chat server 108) in the language of theirchoice (e.g., according to the user language preferences/settings).Additionally, the chat client GUI module 406 may be configured toreceive chat input from the user in the language of their choice (e.g.,according to the user language preferences/settings). As describedherein, the language used in presenting and receiving the chat dialogueat the chat client system 104 may be different from the language used inpresenting and receiving the chat dialogue at another chat clientsystem. More regarding the chat client GUI module 406 is discussed withrespect to FIG. 7.

FIG. 5 is a flowchart illustrating an exemplary method 500 formulti-lingual communication in accordance with various embodiments. Asdescribed below, for some embodiments, the method illustrated by themethod 500 may perform operations in connection with the chat clientsystem 104-1, the chat client system 104-2, the CTT system 114 (e.g., ofthe chat server 108), and the translation module 116 (e.g., oftranslation server 110).

The method 500 may start at operation 502, the language module 204 (ofthe CTT system 114) may being by identifying a first language being usedby a user at a first chat client system (e.g., 104-1) and a secondlanguage being used by a user at a second chat client system (e.g.,104-2). According to some embodiments, the language module 204 mayidentify the first language and the second language by obtaininglanguage preferences/settings from the respective chat client system104.

At operation 504, the CTT communications module 204 (of the CTT system114) may receive an initial message in the first language. In someembodiments, the CTT communications module 204 may receive the initialmessage from the chat host system 112, which may have received theinitial message from a chat client system (e.g., 104-1).

At operation 506, the CTT control module 202 (of the CTT system 114) mayquery the translation data store 210 for a corresponding message in thesecond language that corresponds to the initial message. At operation508, the CTT control module 202 may determine if a corresponding messageis found in the translation data store 210. If one exists, at operation510, the CTT communications module 204 may assist in sending thecorresponding message to the second chat client system (e.g., the chatclient system 104-2). In some embodiments, the corresponding message maybe sent to the chat host system 112, which may relay the correspondingmessage to the second chat client system (e.g., 104-2). The method 500may then end.

If a corresponding message does not exist in the translation data store210, at operation 512, the transformation module 208 may attempt totransform at least a portion of the initial message to a transformedmessage in the first language. As described herein, the message thatresults from the transformation module 208 may be transformed or mayremain unchanged (e.g., when transformation operations of thetransformation module 208 are not applied to the initial message). Forsome embodiments, the transformation module 208 may perform two or moretransformation operations on the initial message in parallel, and inresponse, each of the two or more transformation operations may return aseparate response, from which the transformation module 208 may thenselect one transformed message for further processing (e.g., to be usedin operation 514). Depending on the embodiment, each response maycomprise a flagged text portion, a suggested replacement, or a word orphrase inserted into the initial message. Thereafter, the transformedmessage selected may be according to a priority of selection, which candetermine which transformed message is selected for further processingand according to what precedent. In some embodiments, the priorityselection may be according to which transformation operation is mostlikely to generate a transformed message suitable for a subsequentlookup in the translation data store 210) or for subsequent machinetranslation. Additionally, in some embodiments, the priority ofselection may be according to which transformation operation generatesthe most formal transformed message. The priority of selection maydepend on the transformation/translation strategy selected by theembodiment.

At operation 514, assuming the transformation module 208 transformed themessage, the CTT control module 202 (of the CTT system 114) may querythe translation data store 210 for a corresponding message in the secondlanguage that corresponds to the transformed message. At operation 516,the CTT control module 202 may determine if a corresponding message isfound in the translation data store 210. If one exists, at operation518, the CTT communications module 204 may assist in sending thecorresponding message to the second chat client system (e.g., the chatclient system 104-2). In some embodiments, the corresponding message maybe sent to the chat host system 112, which may then relay thecorresponding message to the second chat client system (e.g., 104-2).The method 500 may then end.

For some embodiments, if a corresponding message still does not exist inthe translation data store 210, at operation 520, the CTT control module202 may determine if there are any additional transformation operationsof the transformation module 208 to perform on the chat message thathave not already been performed.

If an additional transformation operation exists, the method 500 returnsto operation 512 and performs additional transformation operation(s).Depending on the embodiment, the additional transformation operation(s)may involve applying a transform operation different from those alreadyperformed on the initial message by the transformation module 208, mayinvolve applying the same transformation operations performed but todifferent portions of the English chat message, or may involve somecombination thereof. For example, if during the first execution ofoperation 512 the transformation module 208 applies a chatspeak-relatedoperation to the initial message (to create a first transformedmessage), during a second execution of operation 512 the transformationmodule 208 may apply an abbreviation-related operation to the secondtransformed message. Following, a subsequent execution of operation 512,the method 500 may continue to operations 514 and 516, where the CTTcontrol module 202 may re-query the translation data store 210 for acorresponding message in the second language that corresponds to thelatest resulting transformed message, and the CTT control module 202 maydetermine if a corresponding message is found in the translation datastore 210. By performing the transformation and query operations in thisiterative manner, certain embodiments may be able to find acorresponding message before having to perform every transformationoperation available. Those skilled in the art will appreciate that forcertain embodiments, the transformation and query operations may beperformed in series, with the query operation (e.g., operation 514) onlybeing performed after every available transformation operation providedby the transformation module 208 has been performed on the chat message.

If a corresponding message does not exist in the translation data store210 and an additional transformation operation (of the transformationmodule 208) does not exist, at operation 522, (through the translationAPI module 212) the translation module 116 may assist in the translatingthe initial message or the transformed message to a correspondingmessage in the second language. Subsequently, at operation 524, the CTTcommunications module 204 may assist in sending the correspondingmessage to the second chat client system (e.g., the chat client system104-2). According to some embodiments, the corresponding message may besent to the chat host system 112, which may then relay the correspondingmessage to the second chat client system (e.g., 104-2). The method 500may then end.

For certain embodiments, the transformation module 208 may be utilizedto transform the corresponding message in the second language before thecorresponding message is sent to the chat host system 112. As describedherein, the corresponding message may be submitted for furthertransformation processing to further refine the translation for the userat the second chat client system (e.g., 104-2). For example, if theinitial message contains chatspeak in the first language (e.g.,English), additional transformation processing can add, to the extentpossible, chatspeak in the second language.

Though the steps of the above method may be depicted and described in acertain order, those skilled in the art will appreciate that the orderin which the steps are performed may vary between embodiments.Additionally, those skilled in the art will appreciate that thecomponents described above with respect to the method 500 are merelyexamples of components that may be used with the method, and for someembodiments other components may also be utilized in some embodiments.

FIG. 6 is a flowchart illustrating an exemplary method 600 fortransforming communications in accordance with various embodiments. Asdescribed below, for some embodiments, the method 600 may performoperations in connection the transformation module 208 (e.g., of the CTTsystem 114).

The method may start at operation 602, with an initial message beingreceived by the transformation module 208 for transformation processing.Based on some embodiments, the transformation module 208 may receive theinitial message for transformation subsequent to a failure to identify amessage in the translation data store 210, and possibly before theinitial message is machine translated by a third-party or proprietarytranslation process (e.g., the translation module 116, which may beoffered as a cloud-based service). As described herein, thetransformation module 208 may be used in various embodiments tofacilitate or otherwise improve text translation, particularly where thetext comprises a minimal context, brief sentence construction,specialized/domain-related jargon (e.g., chatspeak for Internet-basedchat) abbreviations, acronyms, colloquialisms, proper nouns, commonnouns, profanity, or some combination thereof. Text translations thatmay benefit from the operations of the transformation module 208 mayinclude, without limitation, translations of texts originating fromconversations (e.g., transcript), from offline or online Internet-basedchat (e.g., instant messaging), and from mobile phone messaging (e.g.,SMS or MMS).

At operation 604, the chatspeak module 302 may identify one or morewords or phrases in the initial message that are associated with chatjargon (i.e., chatspeak), and may further suggest replacement (e.g.,corresponding formal/i.e., non-chatspeak) words or phrases for theidentified words or phrases. In some embodiments, the chatspeak module302 may flag an identified chatspeak word or phrase to be skipped orotherwise ignored during a subsequent machine translation (e.g., by thetranslation module 116). Additionally, in some embodiments, anidentified chatspeak word or phrase may be flagged for later review anddisposition by a human operator (e.g., an administrator of the CTTsystem 114). In order to identify a chatspeak word or phrase and/or itscorresponding (formal) word or phrase, some embodiments may utilize adataset (e.g., stored on a data store) comprising chatspeak words orphrases and/or mappings between chatspeak words or phrases and theircorresponding words and phrases. The dataset may be constructed by wayof training or a learning system, may be proprietary (e.g., manuallycollected “in-house” by an administrator of the CTT system 114), may becommercially acquired, or may be derived from a publicly availableInternet knowledgebase. The message resulting from operation 604(hereafter, “the first intermediate message”) may comprise a chatspeakword or phrase flagged by the chatspeak module 302 to be ignored, asuggested replacement, or a non-chatspeak word or phrase inserted intothe initial message by the chatspeak module 302 (e.g., in place of theidentified chatspeak word or phrase). In some instances, the firstintermediate message may be the same as the initial message (e.g., whenno replacement is performed by the chatspeak module 302). Depending onthe embodiment, the first intermediate message that results from thechatspeak module 302 may be provided to another transformation module(in the transformation module 208) for further processing or thesuggested replacement may be provided to the CTT control module 202 todetermine if the message transformed by the chatspeak module 302 is inthe data store 210. Following operation 604, the first intermediatemessage may be provided to the next operation (e.g., operation 606) ofthe transformation module 208 for processing.

At operation 606, the acronym module 304 may identify one or moreacronyms in a chat message, and may further suggest replacement words orphrases corresponding to (e.g., represented by) the acronyms. In someembodiments, the acronym module 304 may flag an identified acronym to beskipped or otherwise ignored during a subsequent machine translation(e.g., by the translation module 116). Additionally, in someembodiments, an identified acronym may be flagged for later review anddisposition by a human operator (e.g., an administrator of the CTTsystem 114). In order to identify an acronym and/or its correspondingword or phrase, some embodiments may utilize a dataset (e.g., stored ona data store) comprising acronyms and/or mappings between acronyms andtheir corresponding words and phrases. The dataset may be constructed byway of training or a learning system, may be proprietary (e.g., manuallycollected “in-house” by an administrator of the CTT system 114), may becommercially acquired, or may be derived from a publicly availableInternet knowledgebase. The message resulting from operation 606(hereafter, “the second intermediate message”) may comprise an acronymflagged by the acronym module 304 to be ignored, a suggestedreplacement, or a word or phrase inserted into the message by theacronym module 304 (e.g., in place of the identified acronym). In someinstances, the second intermediate message may be the same as the firstintermediate message (e.g., when no replacement is performed by theacronym module 304). Depending on the embodiment, the secondintermediate message that results from the acronym module 304 may beprovided to another transformation module (in the transformation module208) for further processing or the suggested replacement may be providedto the CTT control module 202 to determine if the message transformed bythe acronym module 304 is in the data store 210. Following operation606, the second intermediate message may be provided to the nextoperation (e.g., operation 608) of the transformation module 208 forprocessing.

At operation 608, the proper noun module 306 may identify one or moreproper nouns in a chat message, and may further suggest replacementwords or phrases corresponding to (e.g., represented by) the propernouns. In some embodiments, the proper noun module 306 may flag anidentified proper noun to be skipped or otherwise ignored during asubsequent machine translation (e.g., by the translation module 116).Additionally, in some embodiments, an identified proper noun may beflagged for later review and disposition by a human operator (e.g., anadministrator of the CTT system 114). In order to identify a proper nounand/or its corresponding word or phrase, some embodiments may utilize adataset (e.g., stored on a data store) comprising proper nouns (e.g.,well-known proper nouns such as Disneyland®, or common names forindividuals) and/or mappings between proper nouns and theircorresponding words and phrases. The dataset may be constructed by wayof training or a learning system, may be proprietary (e.g., manuallycollected “in-house” by an administrator of the CTT system 114), may becommercially acquired, or may be derived from a publicly availableInternet knowledgebase. The message resulting from operation 608(hereafter, “the third intermediate message”) may comprise a proper nounflagged by the proper noun module 306 to be ignored, a suggestedreplacement, or a word or phrase inserted into the message by the propernoun module 306 (e.g., in place of the identified proper noun). In someinstances, the third intermediate message may be the same as the secondintermediate message (e.g., when no replacement is performed by theproper noun module 306). Depending on the embodiment, the thirdintermediate message that results from the proper noun module 306 may beprovided to another transformation module (in the transformation module208) for further processing or the suggested replacement may be providedto the CTT control module 202 to determine if the message transformed bythe proper noun module 306 is in the data store 210. Following operation608, the third intermediate message may be provided to the nextoperation (e.g., operation 610) of the transformation module 208 forprocessing.

At operation 610, the common noun module 308 may identify one or morecommon nouns in a chat message, and may further suggest replacementwords or phrases corresponding to (e.g., represented by) the commonnouns. In some embodiments, the common noun module 308 may flag anidentified common noun to be skipped or otherwise ignored during asubsequent machine translation (e.g., by the translation module 116).Additionally, in some embodiments, an identified common noun may beflagged for later review and disposition by a human operator (e.g., anadministrator of the CTT system 114). In order to identify a common nounand/or its corresponding word or phrase, some embodiments may utilize adataset (e.g., stored on a data store) comprising common nouns and/ormappings between common nouns and their corresponding words and phrases.The dataset may be constructed by way of training or a learning system,may be proprietary (e.g., manually collected “in-house” by anadministrator of the CTT system 114), may be commercially acquired, ormay be derived from a publicly available Internet knowledgebase. Themessage resulting from operation 610 (hereafter, “the fourthintermediate message”) may comprise a common noun flagged by the commonnoun module 308 to be ignored, a suggested replacement, or a word orphrase inserted into the message by the common noun module 308 (e.g., inplace of the identified common noun). In some instances, the fourthintermediate message may be the same as the third intermediate message(e.g., when no replacement is performed by the common noun module 308).Depending on the embodiment, the fourth intermediate message thatresults from the common noun module 308 may be provided to anothertransformation module (in the transformation module 208) for furtherprocessing or the suggested replacement may be provided to the CTTcontrol module 202 to determine if the message transformed by the commonnoun module 308 is in the data store 210. Following operation 610, thefourth intermediate message may be provided to the next operation (e.g.,operation 612) of the transformation module 208 for processing.

At operation 612, the colloquialism module 310 may identify one or morecolloquial words or phrases in a chat message, and may further suggestreplacement (e.g., corresponding formal/i.e., non-colloquial) words orphrases for the identified words or phrases. In some embodiments, thecolloquialism module 310 may flag an identified colloquial word orphrase to be skipped or otherwise ignored during a subsequent machinetranslation (e.g., by the translation module 116). Additionally, in someembodiments, an identified colloquial word or phrase may be flagged forlater review and disposition by a human operator (e.g., an administratorof the CTT system 114). In order to identify a colloquial word or phraseand/or its corresponding (formal) word or phrase, some embodiments mayutilize a dataset (e.g., stored on a data store) comprising colloquialwords or phrases and/or mappings between colloquial words or phrases andtheir corresponding words and phrases. The dataset may be constructed byway of training or a learning system, may be proprietary (e.g., manuallycollected “in-house” by an administrator of the CTT system 114), may becommercially acquired, or may be derived from a publicly availableInternet knowledgebase. The message resulting from operation 612(hereafter, “the fifth intermediate message”) may comprise a colloquialword or phrase flagged by the colloquialism module 310 to be ignored, asuggested replacement, or a non-colloquial word or phrase inserted intothe message by the colloquialism module 310 (e.g., in place of theidentified colloquial word or phrase). In some instances, the fifthintermediate message may be the same as the fourth intermediate message(e.g., when no replacement is performed by the colloquialism noun module310). Depending on the embodiment, the fifth intermediate message thatresults from the colloquialism module 310 may be provided to anothertransformation module (in the transformation module 208) for furtherprocessing or the suggested replacement may be provided to the CTTcontrol module 202 to determine if the message transformed by thecolloquialism module 310 is in the data store 210. Following operation612, the fifth intermediate message may be provided to the nextoperation (e.g., operation 614) of the transformation module 208 forprocessing.

At operation 614, the spelling check module 312 may identify one or moremisspelled words or phrases in a chat message, and may further suggestreplacement (e.g. corrected) words or phrases for the identified wordsor phrases. For example, the spelling check module 312 may automaticallycorrect the words or phrases with the suggested replacement words orphrases. In some embodiments, the spelling check module 312 may flag anidentified misspelled word or phrase to be skipped or otherwise ignoredduring a subsequent machine translation (e.g., by the translation module116). Additionally, in some embodiments, an identified misspelled wordor phrase may be flagged for later review and disposition by a humanoperator (e.g., an administrator of the CTT system 114). In order toidentify a misspelled word or phrase and/or its corresponding(corrected) word or phrase, some embodiments may utilize a dataset(e.g., stored on a data store) comprising misspelled words or phrasesand/or mappings between misspelled words or phrases and theircorresponding words and phrases. The dataset may be constructed by wayof training or a learning system, may be proprietary (e.g., manuallycollected “in-house” by an administrator of the CTT system 114), may becommercially acquired, or may be derived from a publicly availableInternet knowledgebase. The message resulting from operation 614(hereafter, “the sixth intermediate message”) may comprise a misspelledword or phrase flagged by the spelling check module 312 to be ignored, asuggested replacement, or a corrected word or phrase inserted into themessage by the spelling check module 312 (e.g., in place of themisspelled word or phrase). In some instances, the sixth intermediatemessage may be the same as the fifth intermediate message (e.g., when noreplacement is performed by the spelling check module 312). Depending onthe embodiment, the sixth intermediate message that results from thespelling check module 312 may be provided to another transformationmodule (in the transformation module 208) for further processing or thesuggested replacement may be provided to the CTT control module 202 todetermine if the message transformed by the spelling check module 312 isin the data store 210. Following operation 614, the sixth intermediatemessage may be provided to the next operation (e.g., operation 616) ofthe transformation module 208 for processing.

At operation 616, the abbreviation module 314 may identify one or moreabbreviations in a chat message, and may further suggest replacementwords or phrases corresponding to (e.g., represented by) theabbreviations. In some embodiments, the abbreviation module 314 may flagan identified abbreviation to be skipped or otherwise ignored during asubsequent machine translation (e.g., by the translation module 116).Additionally, in some embodiments, an identified abbreviation may beflagged for later review and disposition by a human operator (e.g., anadministrator of the CTT system 114). In order to identify anabbreviation and/or its corresponding word or phrase, some embodimentsmay utilize a dataset (e.g., stored on a data store) comprisingabbreviations and/or mappings between abbreviations and theircorresponding words and phrases. The dataset may be constructed by wayof training or a learning system, may be proprietary (e.g., manuallycollected “in-house” by an administrator of the CTT system 114), may becommercially acquired, or may be derived from a publicly availableInternet knowledgebase. The message resulting from operation 616(hereafter, “the seventh intermediate message”) may comprise anabbreviation flagged by the abbreviation module 314 to be ignored, asuggested replacement, or a word or phrase inserted into the message bythe abbreviation module 314 (e.g., in place of the identifiedabbreviation). In some instances, the seventh intermediate message maybe the same as the sixth intermediate message (e.g., when no replacementis performed by the abbreviation module 314). Depending on theembodiment, the seventh intermediate message that results from theabbreviation module 314 may be provided to another transformation module(in the transformation module 208) for further processing or thesuggested replacement may be provided to the CTT control module 202 todetermine if the message transformed by the abbreviation module 314 isin the data store 210. Following operation 616, the seventh intermediatemessage may be provided to the next operation (e.g., operation 618) ofthe transformation module 208 for processing.

At operation 618, the profanity module 316 may identify one or moreprofane words or phrases (hereafter, referred to as a “profanity”) in achat message, and may further suggest replacement words or phrases(e.g., suitable substitute) corresponding to the profanity (e.g., atoned down euphemism). In some embodiments, the profanity module 316 mayflag identified profanity to be skipped or otherwise ignored during asubsequent machine translation (e.g., by the translation module 116).Additionally, in some embodiments, identified profanity may be flaggedfor later review and disposition by a human operator (e.g., anadministrator of the CTT system 114). In order to identify profanityand/or its corresponding word or phrase, some embodiments may utilize adataset (e.g., stored on a data store) comprising profanity and/ormappings between abbreviations and their corresponding words andphrases. The dataset may be constructed by way of training or a learningsystem, may be proprietary (e.g., manually collected “in-house” by anadministrator of the CTT system 114), may be commercially acquired, ormay be derived from a publicly available Internet knowledgebase. Themessage resulting from operation 618 (hereafter, “the eighthintermediate message”) may comprise profanity flagged by the profanitymodule 316 to be ignored, a suggested replacement, or a word or phraseinserted into the message by the profanity module 316 (e.g., in place ofthe identified profanity). In some instances, the eighth intermediatemessage may be the same as the seventh intermediate message (e.g., whenno replacement is performed by the profanity module 316). Depending onthe embodiment, the eighth intermediate message that results from theprofanity module 316 may be provided to another transformation module(in the transformation module 208) for further processing or thesuggested replacement may be provided to the CTT control module 202 todetermine if the message transformed by the profanity module 316 is inthe data store 210. Following operation 618, the eighth intermediatemessage may be provided to the next operation of the transformationmodule 208 for processing. The method 600 may then end.

In accordance with some embodiments, the message that ultimately resultsfrom the transformation module 208 (e.g., the eighth intermediatemessage resulting from operation 618) may subsequently be used to querythe translation data store 210 for a corresponding message, which canserve as a translation for the resulting message. Those skilled in theart will appreciate that in some instances, the message resulting fromthe transformation module 208 (e.g., message subsequently used in thequery to the translation data store 210) may be the same as the initialmessage received (e.g., at operation 602) when no transformation hasbeen applied to the initial message (e.g., the initial message passesthrough operations 604-618 without any transformations being applied).

Those skilled in the art will also appreciate that various embodimentsmay perform more or less operations than the ones shown, may performoperations different from those shown, and may perform operations in adifferent order. Generally, the types of transformation operationsperformed, and the order in which they are performed, may be depend ontransformation strategy employed by the embodiments. As noted herein,various embodiments may implement different transformation/translationstrategies in achieving their respective translations, with certainstrategies being well suited for particular translation applications ortranslation contexts. The transformation/translation strategy employedmay determine which transformation operations are performed, when thetransformation operations are performed, or in what order thetransformation operations are performed. The transformation/translationstrategy may also determine what translations are populated into atranslation data stores, and when a translation data store is utilizedin the overall transformation/translation process.

For some embodiments, the intermediate messages resulting fromoperations in the method 600 may have an impact and/or cascading effecton messages that result from subsequent operations in the method 600.Additionally, for some embodiments, when a chat message is processed bythe method 600, each operations of flow chart 600 may be performed onthe chat message before the method concludes. Alternatively, for someembodiments, the method of flowchart 600 may terminate early upon theperformance of only a subset of the operations shown (e.g., after atleast one operation results in a transformation of the chat message).According to some embodiments, the performance of each operation inflowchart 500 may be followed by a query to the translation data store210 for a corresponding message in the desired language based on thelatest resulting transformed message; in the event a correspondingmessage is identified, the method of flowchart 500 may terminate early.

For various embodiments, the method 600 may perform operations 604-612in parallel. For example, the CTT control module 202 may submit theinitial message to two or more operations 604-612 in parallel, andreceive from each of those two or more operations a separate response.Each response may comprise a flagged text portion, a suggestedreplacement, or a word or phrase inserted into the initial message.Thereafter, the CTT control module 202 may select one of the receivedresponses for subsequent processing (e.g., query a translation datastore 210 or translating by the translation module 116), possiblyaccording to a priority of selection (e.g., which can determine whichtransformed message is selected for further processing and according towhat precedent).

For instance, during the method 600, the CTT control module 202 maysubmit an initial message to operation 604 for identifying chatspeakprocessing, operation 610 for common noun processing, and operation 616for abbreviation processing. In response, operation 604 may return theinitial message transformed for chatspeak, operation 610 may return theinitial message unchanged, and operation 616 may return the initialmessage transformed for abbreviations. Subsequently, based on a priorityof selection, the CTT control module 202 may select the transformedmessage returned from operation 616 for further processing.

For certain embodiments, a time limit may be enforced on performingvarious operations in the method 600. The time limit may cause atransformation operation of method 600 to stop performing if aresponse/result is not received before the time limit has expired. Indoing so, various embodiments may ensure that certain transformationoperations do not unnecessarily hinder the overalltransformation/translation process.

Though the operations of the above method may be depicted and describedin a certain order, those skilled in the art will appreciate that theorder in which the operations are performed may vary betweenembodiments. Additionally, those skilled in the art will appreciate thatthe components described above with respect to the method of theflowchart 600 are merely examples of components that may be used withthe method, and for some embodiments other components may also beutilized in some embodiments.

FIG. 7 is a diagram 700 illustrating an exemplary multi-lingual chatsession, between chat client systems 104 (e.g., 104-1 and 104-2), inaccordance with various embodiments. As shown, the chat client system104-1 may comprise a chat client GUI module 406-1, and the chat clientsystem 104-2 may comprise a chat client GUI module 406-2. As describedherein, each of the chat client GUI modules 406-1 and 406-2 may beconfigured to respectively provide users at chat client systems 104-1and 104-2 with graphical input/output access to chat session sharedbetween them. For some embodiments the chat client GUI modules 406-1 and406-2 may present their respective user with a client GUI adapted forreceiving user interactions with respect to the chat dialogue sent andreceived.

As chat dialogue 712 (represented by two-way arrow in FIG. 7) is passedbetween the chat client systems 104-1 and 104-2, the chat client GUImodules 406-1 and 406-2 may present the chat dialogue 712 in thelanguage (implicitly or explicitly) chosen by the user at theirrespective chat client system 104-1 or 104-2. As shown, the chat clientGUI module 406-1 may comprise a chat dialogue box 702 configured topresent the chat dialogue 712 in a first language (e.g., English) in anoutput area 708 and to receive chat input in the first language in asecond area 710. The chat client GUI module 406-2 may comprise a chatdialogue box 714 configured to present the chat dialogue 712 in a secondlanguage (e.g., French) in an output area 720 and to receive chat inputin the second language in a second area 722. For some embodiments, whenthe chat dialogue 712 is presented in the dialogue boxes 702 and 714, itmay include the presentation of usernames (e.g., user online identifier)associated with the users entering the chat messages in the chatdialogue 712.

In the illustrated embodiment of FIG. 7, the language chosen for thechat client system 104-1 is English and the language chosen for the chatclient system 104-2 is French. Accordingly, chat messages 704 (“LOL”)and 706 (“Who u laughin at?”) are presented in English in the dialoguebox 702 of the chat client GUI module 406-1, while their respectivecounterpart chat messages 716 (“MDR”) and 718 (“Qui to fair rire?”) arepresented in French in the dialogue box 714 of the chat client GUImodule 406-2. The translation of the chat messages 704, 706, 716, and718 may be facilitated through various systems and methods describedherein. More regarding the translation of messages similar to chatmessages 704, 706, 716, and 718 are discussed with respect to FIG. 8-10.

FIG. 8 is a flowchart illustrating operation of an exemplarymulti-lingual communication method 800 in accordance with variousembodiments. As described below, for some embodiments, the method 800may perform operations in connection with the chat client system 104-1,the chat client system 104-2, and the CTT system 114 (e.g., of the chartserver 108). In particular, FIG. 8 illustrates the translation of anEnglish chat message comprising the text “LOL” to a French chat messagein accordance with some embodiments. Such a situation may arise when thelanguage being used by the user at the first chat client system 104-1 isEnglish and the language being used by the user at the second chatclient system 104-2 is French. According to some embodiments, and theCTT system 114 may automatically detect these languagechoices/preferences for the chat client systems 104-1 and 104-2.

As shown, at operation 802, the first chat client system 104-1 maysubmit the English message for transmission to the second chat clientsystem 104-2 (e.g., via the chat host system 112). The English messagemay be routed to the CTT control module 202 of the CTT system 114 fortranslation processing.

At operation 804, the CTT control module 202 may query the translationdata store 210 for a chat message that corresponds to the English chatmessage (“LOL”) and that is pre-translated to French. In response, atoperation 806, the translation data store 210 may return to the CTTcontrol module 202 a corresponding French message (“MDR”) thatcorresponds to the English chat message (“LOL”). Subsequently, atoperation 808, the CTT control module 202 may assist in the transmissionof the corresponding French chat message (“MDR”) to the second chatclient system 104-2 (e.g., CTT system 114 submits the correspondingFrench chat message to the chat host system 112 for transmission).

FIG. 9 is a flowchart illustrating operation of an exemplarymulti-lingual communication method 900 in accordance with variousembodiments. As described below, for some embodiments, the methodillustrated by the flowchart 900 may perform operations in connectionwith the chat client system 104-1, the chat client system 104-2, the CTTsystem 114 (e.g., of the chart server 108), and the translation module116 (e.g., of translation server 110). In particular, FIG. 9 illustratesthe translation of an English chat message comprising the text “LOL” toa French equivalent chat message, in accordance with some embodiments.Unlike the illustrated embodiment of FIG. 8, FIG. 9 illustrates theusage of the transformation module 208 (e.g., of the CTT system 114) andthe translation module 116.

As shown, at operation 902, the first chat client system 104-1 maysubmit the English chat message for transmission to the second chatclient system 104-2 (e.g., via the chat host system 112) with a userthat speaks French. The English chat message may be routed to the CTTcontrol module 202 of the CTT system 114 for translation processing.

At operation 904, the CTT control module 202 may query the translationdata store 210 for a French equivalent chat message that corresponds tothe English chat message (“LOL). In response, at operation 906, thetranslation data store 210 may return a query failure to the CTT controlmodule 202 to indicate that the translation data store 210 does not havea corresponding French chat message for the English chat message(“LOL”). If such is the case, at operation 908, the CTT control module202 may submit the English chat message to the transformation module 208for transformation processing in accordance with certain embodiments. Asdescribed herein, the transformation module 208 may comprise multipletransformation-related modules 932 configured to transform a chatmessage to a message more suitable for further translation processing.

At operation 910, the chatspeak module 302 of the transformation module208 may transform the English chat message (“LOL”) to the transformedEnglish chat message (“laugh out loud”), and may return the transformedEnglish chat message to the CTT control module 202 for furtherprocessing. Those skilled in the art will appreciate that, for someembodiments, the English chat message may be processed by additionalmodules of the transformation module 208 before the transformed Englishchat message is returned to the CTT control module 202.

At operation 912, the CTT control module 202 may query the translationdata store 210 for a French equivalent chat message that corresponds tothe transformed English chat message (“laugh out loud”). In response, atoperation 914, the translation data store 210 may return a query failureto the CTT control module 202 to indicate that the translation datastore 210 does not have a corresponding French chat message for thetransformed English chat message (“laugh out loud”). If such is thecase, at operation 916, the CTT control module 202 may submit thetransformed English chat message to the translation module 116 formachine translation processing in accordance with certain embodiments.

At operation 918, the translation module 116 may return amachine-translated French chat message (“mort de rire”) that correspondsto the transformed English chat message. The resultingmachine-translated French chat message (“mort de rire”) is an example ofa transformed translation of an English chatspeak chat message (“LOL”).

At operation 920, the CTT control module 202 may submit themachine-translated French chat message (“mort de rire”) to thetransformation module 208 for further transformation processing of themachine-translated French chat message in accordance with certainembodiments. As noted herein, the machine-translated text may besubmitted for further transformation processing to further refine theFrench translation. For example, if the original English chat messagecontained English chatspeak, the additional transformation processingcan add, to the extent possible, French chatspeak. Accordingly, atoperation 922, the chatspeak module 302 of the transformation module 208may transform the machine-translated French chat message (“mort derire”) to the transformed French chat message (“MDR”), and may returnthe transformed French chat message to the CTT control module 202 forfurther processing.

Eventually, at operation 924, the CTT control module 202 may assist inthe transmission of the corresponding French chat message (“MDR”) to thesecond chat client system 104-2 (e.g., CTT system 114 submits thecorresponding French chat message to the chat host system 112 fortransmission). Additionally, at operation 926, the CTT control module202 may store a translation mapping in the translation data store 210 ofthe transformed translation between the original English chat message(“LOL”) and the translated French chat message (“MDR”). Once the mappingis stored in the translation data store 210, it may be used to storetranslation entries to speed up future translations, e.g., asillustrated in FIG. 8. As noted herein, the translation data store 210may store mappings of transformed translations and untransformedtranslations.

For some embodiments, the CTT control module 202 may also storeequivalent (transformed and untransformed) translation mappingsdetermined during the operation of the method 900. For certainembodiments, the translation mappings may be between chat message thatwere not original located in the translation data store 210 (e.g., thechat message shown for operation 904, and the chat message shown foroperation 912) and a corresponding message determined during operationssubsequent to the translation data store 210 lookups (e.g., a mappingbetween a query to the translation data store 210 that returns no resultand a corresponding chat message determined after the query, by way ofthe transformation module 208 and/or the translation module 116).

For instance, as shown in FIG. 9, the CTT control module 202 queries thetranslation data store 210 for original English chat message (“LOL” atoperation 904 and the transformed English chat message (“laugh outloud”) at operation 912, both of which resulted in the CTT controlmodule 202 receiving no results from the translation data store 210 (atoperations 906 and 914, respectively). However, at operation 916, theCTT control module 202 eventually submits the transformed Englishmessage (“laugh out loud”) to the machine translation module 116 formachine translation and receives, in response the machine-translatedFrench chat message (“mort de rire”) at operation 918. Accordingly, atoperation 928, the CTT control module 202 may store a translationmapping in the translation data store 210 of the transformed translationbetween the original English chat message (“LOL”) and themachine-translated French chat message (“mort de rire”). Likewise, atoperation 930, the CTT control module 202 may store a translationmapping in the translation data store 210 of the transformed translationbetween the transformed English chat message (“laugh out loud”) and themachine-translated French chat message (“mort de rire”). In doing so,next time method 900 queries the translation data store 210 for theoriginal English chat message (“LOL”) or the transformed English chatmessage (“laugh out loud”), the translation data store 210 will providethe corresponding transformed translations.

FIG. 10 is a flowchart illustrating operation of an exemplarymulti-lingual communication method 1000 in accordance with variousembodiments. As described below, for some embodiments, the method 1000may perform operations in connection with the chat client system 104-1,the chat client system 104-2, the CTT system 114 (e.g., of the chartserver 108), and the translation module 116 (e.g., of the translationserver 110). In particular, FIG. 10 illustrates the translation of anEnglish chat message comprising the text “Who u laughin at?” to a Frenchchat message, in accordance with some embodiments.

As shown, at operation 1002, the first chat client system 104-1 maysubmit the English chat message for transmission to the second chatclient system 104-2 (e.g., via the chat host system 112). The Englishchat message may be routed to the CTT control module 202 of the CTTsystem 114 for translation processing.

At operation 1004, the CTT control module 202 may query the translationdata store 210 for a French equivalent chat message that corresponds tothe English chat message (“Who u laughin at?”). In response, atoperation 1006, the translation data store 210 may return a queryfailure to the CTT control module 202 to indicate that the translationdata store 210 does not have a corresponding French chat message for theEnglish chat message (“Who u laughin at?”). If such is the case, atoperation 1008, the CTT control module 202 may submit the English chatmessage to the transformation module 208 for transformation processingin accordance with certain embodiments. As described herein, thetransformation module 208 may comprise multiple transformation-relatedmodules 1036 configured to transform a chat message to a message moresuitable for further translation processing.

At operation 1010, the chatspeak module 302 of the transformation module208 may transform the English chat message (“Who u laughin at?”) to thetransformed English chat message (“Who you laughin at?”), and pass onthe transformed English chat message to additional modules of thetransformation module 208 for further processing, such as the spellingcheck module 312.

As discussed herein, various modules of transformation module 208,including the chatspeak module 302, may be configured to identify one ormore words or phrases in a chat message and suggest replacement words orphrases for the identified words or phrases. Accordingly, those skilledin the art would appreciate that for some embodiments, thetransformation performed/suggested by a module of transformation module208 may involve a word-to-phrase or a phrase-to-phrase transformation ofthe chat message. For example, at operation 1010, the chatspeak module302 may alternatively transform the English chat message (“Who u laughinat?”) to the transformed English chat message (“Who are you laughingat?”), possibly by replacing/suggesting the replacement of the phrase“who u” with “who are you” during the transformation (followed by thereplacement/suggestion of the replacing the word “laughin” with“laughing”). In doing so, various modules of the transformation module208, such as the chatspeak module 302, may provide grammaticalimprovements to their respective transformations, while possiblyobviating the need for a separate module in the transformation module208 to implement grammar improvements.

For some embodiments, before the transformed English chat message ispassed on to additional modules of the transformation module 208, thechatspeak module 302 may pass on the transformed English chat message tothe CTT control module 202 at operation 1010. In turn, the CTT controlmodule 202 may query the translation data store 210 (at operation 1012)for a French equivalent chat message that corresponds to the transformedEnglish chat message (“Who you laughin at?”). In response, at operation1014, the translation data store 210 may return a query failure to theCTT control module 202 to indicate that the translation data store 210does not have a corresponding French chat message for the transformedEnglish chat message (“Who you laughin at?”).

At operation 1016, the spelling check module 312 may perform a spellcheck process on the transformed English chat message (“Who you laughinat?”) at operation 1018. During the spell check process, the spellingcheck module 312 may correct the transformed English chat message to acorrected English chat message (“Who you laughing at?”), and may returnthe corrected English chat message to the CTT control module 202. Thoseskilled in the art will appreciate that for some embodiments, thecorrected English chat message may processed by additional modules ofthe transformation module 208 before the transformed English chatmessage is returned to the CTT control module 202.

At operation 1020, the CTT control module 202 may query the translationdata store 210 for a French equivalent chat message that corresponds tothe corrected English chat message (“Who you laughing at?”). Inresponse, at operation 1022, the translation data store 210 may return aquery failure to the CTT control module 202 to indicate that thetranslation data store 210 does not have a corresponding French chatmessage for the corrected English chat message (“Who you laughing at?”).If such is the case, at operation 1024, the CTT control module 202 maysubmit the corrected English chat message to the translation module 116for machine translation processing in accordance with certainembodiments.

At operation 1026, the translation module 116 may return amachine-translated French chat message (“Qui te fait rire?”) thatcorresponds to the corrected English chat message. At operation 1028,the CTT control module 202 may submit the machine-translated French chatmessage (“Qui te fait rire?”) to the transformation module 208 forfurther transformation processing of the machine-translated French chatmessage in accordance with certain embodiments.

As noted herein, the machine-translated text may be submitted forfurther transformation processing to further refine the translation ofthe text. For example, if the original English chat message containedEnglish chatspeak, the additional transformation processing can add, tothe extent possible, French chatspeak. At operation 1030, thetransformation module 208 may return the machine-translated French chatmessage (“Qui te fait rire?”) unchanged to the CTT control module 202for further processing (e.g., when the modules of the transformationmodule 208 do not apply any changes to the machine-translated Frenchchat message).

At operation 1032, the CTT control module 202 may assist in thetransmission of the machine-translated French chat message (“Qui te faitrire?”) to the second chat client system 104-2 (e.g., CTT system 114submits the corresponding French chat message to the chat host system112 for transmission). Additionally, at operation 1034, the CTT controlmodule 202 may store a translation mapping in the translation data store210 between the original English chat message (“Who u laughin at?”) andthe translated French chat message (“Qui te fait rire?”). As describedherein, in additional operations (not shown), the CTT control module 202may also store equivalent translation mappings in the translation datastore 210 based on previously failed queries to the translation datastore 210 and corresponding messages determined subsequent to thosequeries (e.g., similar to operations 928 and 930 in FIG. 9).

According to some embodiments, the transformation operations performedby the transformation module 208 may comprise performing certaintransformation operations in parallel, and perform certaintransformation operations in serial. The order in which transformationoperations are performed in parallel and in serial may vary betweenvarious embodiments. As described herein, where the transformationoperations are performed in parallel, some embodiments may employ apriority of selection to determine which transformed message is selectedfor further processing and according to what precedent.

FIG. 11 is a flowchart illustrating operation of an exemplarymulti-lingual communication method 1100 in accordance with variousembodiments. As described below, for some embodiments, the method 1100may perform operations in connection with the chat client system 104-1,the chat client system 104-2, the CTT system 114 (e.g., of the chartserver 108), and the translation module 116 (e.g., of the translationserver 110). In particular, FIG. 11 illustrates the translation of anEnglish chat message comprising the text “Who u laughin at?” to a Frenchchat message by parallel transformation operations, in accordance withsome embodiments.

As shown, at operation 1102, the first chat client system 104-1 maysubmit the English chat message for transmission to the second chatclient system 104-2 (e.g., via the chat host system 112). The Englishchat message may be routed to the CTT control module 202 of the CTTsystem 114 for translation processing.

At operation 1104, the CTT control module 202 may query the translationdata store 210 for a French equivalent chat message that corresponds tothe English chat message (“Who u laughin at?”). In response, atoperation 1106, the translation data store 210 may return a queryfailure to the CTT control module 202 to indicate that the translationdata store 210 does not have a corresponding French chat message for theEnglish chat message (“Who u laughin at?”).

If such is the case, the CTT control module 202 may submit the Englishchat message to the transformation module 208 for transformationprocessing in accordance with certain embodiments. As described herein,the transformation module 208 may comprise multipletransformation-related modules 1130 configured to transform a chatmessage to a message more suitable for further translation processing.As shown in FIG. 11, during operations 1108, the CTT control module 202may submit the English chat message (“Who u laughin at?”), in parallel,to two or more transformation-related modules 1130 of the transformationmodule 208. Additionally, during operations 1108, the CTT control module202 may be receiving results from the transformation-related modules1130 in parallel, and submitting queries to the translation data store210, based on the transformation results, in parallel.

Accordingly, at operation 1110 a, the CTT control module 202 may submitthe English chat message (“Who u laughin at?”) to the chatspeak module302 for transformation processing. In parallel, at operation 1110 b, theCTT control module 202 may submit the English chat message (“Who ulaughin at?”) to the spelling check module 312 for transformationprocessing. Subsequently, the CTT control module 202 may receive a firsttransformed English chat message (“Who you laughin at?”) from thechatspeak module 302 at operation 1112 a, while at operation 1112 b theCTT control module 202 may receive a second transformed English chatmessage (“Who u laughing at?”) from the spelling check module 312.Depending on their respective transformation processing times, thechatspeak module 302, the spelling check module 312, and the othertransformation-related modules 1130 may respond to the CTT controlmodule 202 in serial or in parallel with respect to one another.

Subsequently, at operation 1114 a, the CTT control module 202 may querythe translation data store 210 for a French equivalent chat message thatcorresponds to the first transformed English chat message (“Who youlaughin at?”). At operation 1114 b, the CTT control module 202 may querythe translation data store 210 for a French equivalent chat message thatcorresponds to the second transformed English chat message (“Who ulaughing at?”). For some embodiments, during operations 1114 a and 1114b, the CTT control module 202 may query the translation data store 210in serial or in parallel. In some embodiments, the timings of thequeries may depend on when the transformation-related modules 1130 ofthe transformation module 208 return their respective responses. Asshown in FIG. 11, the translation data store 210 may return a queryfailure (e.g., <FAIL>) for the queries at operations 1116 a and 1116 b.

Eventually, the CTT control module 202 may select one transformedmessage, from the two or more messages that result from the paralleloperations 1108, for further processing. Where only one of thetransformation-related modules 1130 results in a transformed message,the CTT control module 202 may select that particular transformedmessage for further processing. As noted herein, the CTT control module202 may select a transformed message based on a priority of selection,which may be determined according to the transformation/translationstrategy selected by the embodiments. For some embodiments, the priorityof selection may be based on whether the transformed message has themost formal content, the transformed message has the mosttransformations, or the transformed message results from atransformation-related module known for having a high likelihood ofproducing a transformed message suitable for machine-translation.

Once a transformed message has been selected, at operation 1118, the CTTcontrol module 202 may submit the transformed English chat message tothe translation module 116 for machine translation processing inaccordance with certain embodiments. For example, as shown in FIG. 11,the CTT control module 202 may select the first transformed English chatmessage produced by the chatspeak module 302 (“Who you laughin at?”) forsubmission to the translation module 116.

At operation 1120, the translation module 116 may return amachine-translated French chat message (“Qui te fait rire?”) thatcorresponds to the first transformed English chat message (and despitecomprising the misspelled word “laughin”). At operation 1122, the CTTcontrol module 202 may submit the machine-translated French chat message(“Qui te fait rire?”) to the transformation module 208 for furthertransformation processing of the machine-translated French chat messagein accordance with certain embodiments.

As noted herein, the machine-translated text may be submitted forfurther transformation processing to further refine the translation ofthe text. For example, if the original English chat message containedEnglish chatspeak, the additional transformation processing can add, tothe extent possible, French chatspeak. At operation 1124, thetransformation module 208 may return the machine-translated French chatmessage (“Qui te fait rire?”) unchanged to the CTT control module 202for further processing (e.g., when the modules of the transformationmodule 208 do not apply any changes to the machine-translated Frenchchat message).

At operation 1126, the CTT control module 202 may assist in thetransmission of the machine-translated French chat message (“Qui te faitrire?”) to the second chat client system 104-2 (e.g., CTT system 114submits the corresponding French chat message to the chat host system112 for transmission). Additionally, at operation 1128, the CTT controlmodule 202 may store a translation mapping in the translation data store210 between the original English chat message (“Who u laughin at?”) andthe translated French chat message (“Qui te fait rire?”). As describedherein, in additional operations (not shown), the CTT control module 202may also store equivalent translation mappings in the translation datastore 210 based on previously failed queries to the translation datastore 210 and corresponding messages determined subsequent to thosequeries (e.g., similar to operations 928 and 930 in FIG. 9).

For some embodiments, the transformation operations may be performed ina hybrid serial/parallel arrangement, whereby some transformationoperations are performed in parallel and other transformation operationsare performed in serial. For example, as shown in FIG. 11, the Englishchat message (“Who u laughin at?”) is submitted to the chat speak module302 and spelling check module 312 in parallel at operations 1110 a and1110 b. Subsequently, once one of the resulting transformed messages isselected (e.g., based on a priority of selection), the othertransformation-related modules 1130 of the transformation module 208(e.g., the acronym module 304, the proper noun module 306, and thecommon noun module 308) may operate on the selected transformed messagein parallel.

FIG. 12 is a block diagram of an exemplary digital device 1200. Thedigital device 1200 comprises a processor 1202, a memory system 1204, astorage system 1206, a communication network interface 1208, an I/Ointerface 1210, and a display interface 1212 communicatively coupled toa bus 1214. The processor 1202 is configured to execute executableinstructions (e.g., programs). In some embodiments, the processor 1202comprises circuitry or any processor capable of processing theexecutable instructions.

The memory system 1204 is any memory configured to store data. Someexamples of the memory system 1204 are storage devices, such as RAM orROM. The memory system 1204 can comprise the ram cache. In variousembodiments, data is stored within the memory system 1204. The datawithin the memory system 1204 may be cleared or ultimately transferredto the storage system 1206.

The storage system 1206 is any storage configured to retrieve and storedata. Some examples of the storage system 1206 are flash drives, harddrives, optical drives, and/or magnetic tape. In some embodiments, thedigital device 1200 includes a memory system 1204 in the form of RAM anda storage system 1206 in the form of flash data. Both the memory system1204 and the storage system 1206 comprise computer readable media whichmay store instructions or programs that are executable by a computerprocessor including the processor 1202.

The communications network interface (com. network interface) 1208 canbe coupled to a network (e.g., the computer network 106) via the link1216. The communication network interface 1208 may support communicationover an Ethernet connection, a serial connection, a parallel connection,or an ATA connection, for example. The communication network interface1208 may also support wireless communication (e.g., 802.11 a/b/g/n,WiMax). It will be apparent to those skilled in the art that thecommunication network interface 1208 can support many wired and wirelessstandards.

The optional input/output (I/O) interface 1210 is any device thatreceives input from the user and output data. The optional displayinterface 1212 is any device that is configured to output graphics anddata to a display. In one example, the display interface 1212 is agraphics adapter.

It will be appreciated by those skilled in the art that the hardwareelements of the digital device 1200 are not limited to those depicted inFIG. 12. A digital device 1200 may comprise more or less hardwareelements than those depicted. Further, hardware elements may sharefunctionality and still be within various embodiments described herein.In one example, encoding and/or decoding may be performed by theprocessor 1202 and/or a co-processor located on a GPU (i.e., Nvidia®).

The above-described functions and components can be comprised ofinstructions that are stored on a storage medium such as a computerreadable medium. The instructions can be retrieved and executed by aprocessor. Some examples of instructions are software, program code, andfirmware. Some examples of storage medium are memory devices, tape,disks, integrated circuits, and servers. The instructions areoperational when executed by the processor to direct the processor tooperate in accord with some embodiments. Those skilled in the art arefamiliar with instructions, processor(s), and storage medium.

Various embodiments are described herein as examples. It will beapparent to those skilled in the art that various modifications may bemade and other embodiments can be used without departing from thebroader scope of the invention(s) presented herein. These and othervariations upon the exemplary embodiments are intended to be covered bythe present invention(s).

FIG. 13 is a block diagram illustrating an example user feedback systemin accordance with various embodiments. For illustrative purposes, inFIG. 13 the example system for user feedback is shown as part of anexample communication transformation and translation (CTT) system 1300in accordance with various embodiments. For example, the example userfeedback system may be part of a chat translation system employed byvarious multi-lingual chat systems, including game chat systemsavailable in conjunction with MMO games (e.g., in-game chat system).With use of the example user feedback system, various players of the MMOgame may choose to provide user feedback, for example through thein-game chat system, for flagged words or phrases, possibly in returnfor in-game currency/credit/item as an incentive for thefeedback/approved feedback. Those skilled in the art will appreciatethat for some embodiments, the example user feedback system, and some orall of its related components, may be separate and/or independent fromthe example communication transformation and translation system 1300.

As used herein, “feedback” should be understood to refer to “userfeedback” or “user feedback response,” possibly in response to a queryrequesting feedback for a transformation or a translation. It shouldalso be understood that that user feedback can comprise user feedbackfor a transformation or user feedback for a translation. User feedbackmay comprise a definition for a given word or phrase that: (i) permitsthe given word or phrase to be transformed from the given word or phrasein a first language to a corresponding word or phrase in the (same)first language; (ii) permits the given word or phrase to be transformedfrom the given word or phrase in the first language to a correspondingword or phrase in a (different) second language, thereby performing atransformed translation of the word or phrase; and/or (iii) permits thegiven word or phrase to be translated from a first language to a secondlanguage, thereby performing a untransformed translation of the word orphrase. As described herein, a user feedback response may be received inresponse to a query requesting user feedback in connection with atranslation failure.

As used herein, a “failure of translation” or a “translation failure”may be understood to include an actual failure to translate a text(e.g., translated text is identical to the original text), or texttranslations that are flagged as potentially containing a translationfailure (e.g., flagged by a user reading a translated text and believingthe translated text contains questionable or incorrect translation).

As shown in FIG. 13, the CTT system 1300 may comprise a communicationtransformation and translation (CTT) control module 1302, acommunication transformation and translation (CTT) communications module1304, a language module 1306, a transformation module 1308, atranslation data stores 1310, and a translation application programminginterface (API) module 1312, In some embodiments, the CTT control module1302, the CTT communications module 1304, the language module 1306, thetransformation module 1308, the translation data stores 1310, and thetranslation API module 1312 may be similar to the components of the CTTsystem 114 as described herein.

As also shown in FIG. 13, the CTT system 1300 may comprise a translationfailure management module 1314, a user feedback (UF) query generationmodule 1316, a user feedback (UF) query audience selection module 1318,a query/response valuation module 1320, a query application programinterface (API) module 1322, a response evaluation module 1324, atranslation data store update module 1326, an audience competenceevaluation module 1328, and an incentive reward module 1330.

The translation failure management module 1314 may be configured tofacilitate management of translation failures (e.g., failure totranslate a text from a first language to a second language). For someembodiments, the translation failure management module 1314 may beadapted to handle one or more translation failures that may beexperienced by the CTT system 1300 as the system 1300 attempts totranslate a chat message sent from a first user, who is using a firstlanguage during their chat sessions, to a second user, who is using asecond language during their chat session. As described herein, the textto be translated by the CTT system 1300 may be chat messages, which maycomprise chatspeak, abbreviates, colloquialisms, idioms, and the like.It will be understood that during transformation or translation of achat message, some or all of the chat messages may result in translationfailure, possibly due to the inability of the CTT system 1300 totransform and/or translate those failure causing portions of the chatmessage.

Features provided by the translation failure management module 1314 mayinclude, without limitation: automatically detecting when a failure totranslation of text has occurred; automatically detecting when certainwords or phrases of a text are untranslatable; enabling a user to flagsome or all of a “translated” text as containing actual or potentialtranslation errors after a translation process; managing whattranslations failures are selected for user feedback from variousaudience members (e.g., chat members, such as players using an in-gamechat system); managing what words or phrases associated with atranslation failure are selected for user feedback (e.g., based on howoften the word is encountered/used in text to be translated); andmanaging when a translation failure is no longer a candidate forselection for user feedback (e.g., words or phrases associated with atranslation failure are removed from the selection pool).

In various embodiments, the translation failure management module 1314may manage the overall availability of words or phrases for userfeedback by audience members. As used herein, a “user feedbackopportunity” may comprise a word or phrase available for user feedbackby an audience member. The translation failure management module 1314may throttle or suspend availability of one or more user feedbackopportunities based on various conditions, such as the availability ofcomputing resources for user feedback, incentives rewarded in connectionwith user feedback, quality of feedback response received in the past,and the like. For instance, it may be determined that continuing toreward incentives (e.g., in-game currency or in-game item) for approveduser feedback responses may negatively impact the economy relating tothe incentive (e.g., the in-game economy). Accordingly, some embodimentsmay (temporarily) suspend the availability of user feedbackopportunities that reward the incentives, or adjust the incentivesrewarded (e.g., amount of incentive or type of incentive) for userfeedback opportunities when those embodiments detect that the currentinventive strategy is harmful to an economy of concern (e.g., in-gameeconomy).

As described herein, an audience member from which user feedback issolicited may be one who has volunteered to provide such feedback,possibly as a participant in a user feedback program and/or in returnfor an incentive when their submitted feedback has been approved (e.g.,as being correct, accepted, or useful in defining a transformation ortranslation). For some embodiments, the user feedback systems describedherein may be part of a game system, such as an online MMO game, wherethe audience members are game players that choose to otherwiseparticipate in user feedback opportunities, possibly for incentivesuseful in the game system (e.g., in-game currency or in-game items). Inthis way, the user feedback system may be presented as a game featurethat game playing audience member regard as a “game” to be “played” fora reward, thereby leveraging the competition inherent to the gamingenvironments. More regarding selection of audience members is describedwith respect to the UF query audience selection module 1318.

As described herein, a word or phrase may be flagged by a communicationssystem user that believes that the word or phrase of concern ispreventing a translation or causing an inaccurate translation ofcommunicated text. For example, in a multi-lingual multi-user chatsystem associated with an online game, a game player may flag an entirechat message they have received, or flag certain portions of the chatmessage (e.g., word or phrase thereof), as potentially having atranslation problem or failure. Words or phrases associated with atranslation failure may include, for example, specialized/domain-relatedjargon, abbreviations, acronyms, proper nouns, common nouns,diminutives, colloquial words or phrases, and profane words or phrases.Additionally, the word or phrase may be flagged by a system or methodthat automatically detects the word or phrase as being untranslatable,possibly preventing translation of larger phrases or sentences.

The translation management module 1314 may be responsible for selectinga word or phrase as being the subject of a user feedback, where theselected word or phrase may be flagged as being associated with anactual or potential failure to translate text from a first language to asecond language. The translation management module 1314 may select aword or phrase based on a number of factors, some of which include thecurrent importance of the word or phrase in translations (e.g.,importance based on overall usage of the word or phrase), complexity ofthe word or phrase (e.g., difficulty of the word or phrase, or how longthe word or phrase has been an issue), the competency of the userselected/volunteering to provide user feedback (e.g., the user hascompetency in the second language), and a preference of the userselected/volunteering to provide user feedback. Those skilled in the artwill appreciate other factors for selecting words or phrases for userfeedback may be utilized by various embodiments.

For various embodiments, users selected to provide user feedback may beones volunteering to provide such feedback. For instance, a user maychoose to provide user feedback by setting an associated userpreference, by selecting an in-chat system advertisement that issoliciting user feedback, and/or browsing through a section that listsone or more user feedback opportunities available for selection.

As noted herein, the translation failure management 1314 may manage whena particular word or phrase that is associated with an actual orpotential translation failure is no longer a candidate for selection foruser feedback by audience members. Various conditions can lead thetranslation failure management 1314 to make such a determinationincluding, for instance: when a specific number of feedback responseshas been received in connection with the certain word or phrase or withthe associated translation failure; when a specific number of consistentfeedback responses has been received in connection with the certain wordor phrase or with the associated translation failure; and when a givenfeedback response has been approved as a valid response for the userfeedback sought (e.g., a specific response from a set of unique feedbackresponses has been manually approved by an administrator as a correctresponse).

The UF query generation module 1316 may be configured to generate aquery for obtaining user feedback, from a selected audience member, fora given word or phrase selected for user feedback. As described herein,the audience member selected may be a user who has volunteered toprovide user feedback, possibly in return for an incentive when the userfeedback submitted has been approved as being correct, an acceptedfeedback response, or useful in refining a translation in question. Moreregarding selection of audience members is described with respect to theUF query audience selection module 1318.

For some embodiments, the query generated may include one or moremethods of receiving a query response from the selected audience member.For example, the generated query may include, as a response method, alisting of predefined responses from which the audience member canselect as their response to the generated query (also referred to hereinas a “select-form response”). In another example, the generated querymay include, as a response method, one or more fields configured toreceive as their response to the generated query, a text value enteredinto a field by the audience member. (also referred to herein as a“free-form response”) Other response methods may include a graphicaluser interface (GUI) elements, text values, or some combination thereof.

The one or more response methods included in the generated query may beaccording to a number of factors including, for example: a preference ofthe audience member; the importance and/or complexity of the word orphrase for which user feedback is sought; the number of feedbackresponses received thus far for the word or phrase for which userfeedback is sought; the number of consistent feedback responses receivedthus far for the word or phrase for which user feedback is sought; andwhether there is enough free-form feedback responses from which tocreate a selection-form response. For some embodiments, the audiencemember to respond to the generated query can select from two or moreresponse methods for the generated query.

Additionally, the languages for which an audience member is presenteduser feedback opportunities may be according to a number of factorsincluding, for example: whether the audience member is multi-lingual bymonitoring the audience member's chat history; whether the audiencemember language abilities meet or exceed a particular languageconfidence level; and a language setting on the user device the audiencemember is using to participate in a user feedback opportunity (e.g.,default language setting for the audience member's device). For example,where the default device language of an audience member's device isGerman and he or she selects to participate in a user feedbackopportunity, the query generated and sent to the audience member willbased on the German language (e.g., query for defining an English wordor phrase to a German word or phrase, or vice versa). For someembodiments, the generated query may provide an audience member with anoption to select the language of the response to be submitted (e.g.,when the generated query does not specify or require a specificlanguage) and/or with an option to provide more two or more responseswith each response possibly being of a different language. For some suchembodiments, the audience member may be presented with language responseoptions according to various language factors discussed herein, such aswhether the audience member is multi-lingual and a language setting onthe user device of the audience member.

The UF query audience selection module 1318 may be configured to selectan audience member from which to solicit user feedback. In someembodiments, the UF query audience selection module 1318 may select theaudience member from a pool of audience members who have volunteered toprovide user feedback, possibly in return of incentive (which may bereceived when the user feedback is approved/accepted). As describedherein, an audience member may volunteer to participate in translation,at which time the audience member may be included in the pool ofaudience members from which the UF query audience selection module 1318selects for user feedback solicitation. In some embodiments, when the UFquery audience selection module 1318 selects an audience member, theaudience member may be provided with a query generated by the UF querygeneration module 1316 to obtain user feedback. The query generated bythe UF query generation module 1316 may be provided to the audiencemember selected, by the UF query audience selection module 1318, as partof a listing of user feedbacks available for participation by theselected audience member. Once the generated query is provided to theselected audience member, the selected audience member may initiate auser feedback session in which the generated query is presented to theaudience member (e.g., with one or more response methods associated withthe generated query) and the selected audience member can provide one ormore responses (e.g., by way of one or more response methods includedwith the generated query).

Depending on the embodiment, where a word or phrase is selected for userfeedback before the audience member is selected, the audience member maybe selected from a set of candidate audience members based on theselected word or phrase (e.g., whether the selected word or phrasematches the competency or preferences of the audience member).Alternatively, where the audience member is selected before the word orselected is selected for user feedback, the word or phrase selected maybe based according to the audience member that is selected. Thecompetency of a given audience member may be determined based on one ormore feedback responses previously provided by the given audience member(e.g., in connection with previous queries generated and provided to thegiven audience member) and/or one or more language capabilities of theaudience member. An audience member, for example, may be evaluated tohave requisite competency to provide user feedback for one or more wordsor phrases associated with a translation failure when the audiencemember has achieved a specific number of approved feedback responses. Inanother example, an audience member may be evaluated to have competencyin a specific language based on at least previously submitted feedbackresponses for a specific language, the preferences of the audiencemember, and/or information related to the audience member that indicatestheir fluency in the specific language. In a further example, anaudience member who has previously submitted feedback responses thatwere evaluated as being incorrect or fraudulent (e.g., nonsensical orfabricated responses) may be determined to have lower competency.

For some embodiments, once a given audience member has provided aresponse in connection with a word or phrase (and the response ispossibly approved), the given audience member may not be re-selected forproviding user feedback for the same word or phrase. Certain embodimentsmay make an exception to this restriction when the given audience memberprovides a response in a language that is different from the language ofa response previously submitted for the same word or phrase.

Where incentives are provided in return for user feedback (e.g., uponapproval of a feedback response), various embodiments may selectaudience members based on a quota, such as a time-based quota (e.g.,hourly, daily, monthly, yearly limit for submitting feedback responses)or an incentives-earned quota (e.g., limit of in-game currency or itemsawarded for incentive-based user feedback), where the quota isassociated with individual audience members, groups of audience members,or some combination thereof. For some embodiments, the UF query audienceselection module 1318 may throttle or suspend availability of userfeedback opportunities to one or more audience members based on variousconditions, such as the availability of computing resources for userfeedback, incentives rewarded in connection with user feedback, qualityof feedback response received in the past, and the like.

The query/response (QR) valuation module 1320 may be configured todetermine the value or importance of a query or query response based onthe word or phrase for which the query or query response is solicitinguser feedback. Example of factors considered by various embodiments whendetermining the value a query or a query response may include thecomplexity of the word or phrase (e.g., higher the complexity, higherthe value), the importance of the word or phrase totransformation/translation processes (e.g., higher the importance,higher the value), the response method employed by the query or queryresponse (e.g., higher value for a free-form response method over aselection-form response method), the type of word or phrase (e.g.,chatspeak, abbreviation, or colloquial), or the one or more languagesinvolved (e.g., a query comprising an English word or phrase and thatreceives a French response has higher value than a query comprising anEnglish word or phrase that receives an English response). In someembodiments, an incentive rewarded for a given user feedback may bebased on the value associated with the query or query response. For someembodiments, the QR valuation module 1320 may be configured to determinea value for a query or query response based on the efficacy of previousqueries or query responses in soliciting beneficial, useful, or accurateuser feedback. Accordingly, for some embodiments, the value may bedynamically adjusted based on the latest efficacy of achieved byprevious queries or query responses. In certain embodiments, the QRvaluation module 1320 may enable an administrator (e.g., of the CTTsystem 1300) or another authorized user to manually assign or adjustvalues for queries and/or query responses in connection with one or moreuser feedback.

The query API module 1322 may be configured to facilitate transmittinguser feedback queries to selected audience members and/or receivingquery responses from selected audience members. In some embodiments, thequery API 1322 may be adapted to provide a listing of available userfeedback opportunities available to the selected audience member,provide queries relating to one or more the user feedback opportunitiesselected by the selected audience member, receiving responses for one ormore the user feedback opportunities selected by the selected audiencemember, and/or providing the selected audience member with the currentstatus of query responses submitted for approval thus far (e.g., statusof approved, rejected, or pending/awaiting approval). For someembodiments, the query API 1322 may obtain one or preferences associatedwith a given audience member, possibly from a chat client system beingused by the given audience member to interact with the CTT system 1300.As noted herein, a preference associated with an audience member candetermine selection of the word or phrase for which user feedback issolicited from the audience member, and/or can determine of whether theaudience member is selected to receive a query for user feedback of agiven word or phrase.

The response evaluation module 1324 may be configured to evaluate aquery response submitted in connection with a query generated to obtaina user feedback. For some embodiments, the response evaluation module1324 may evaluate query responses in one or more phases.

During a validation phase, the response evaluation module 1324 maydisposition one or more unique responses, submitted for a givengenerated query, as approved or rejected for the given generated query.In some embodiments, a unique response, submitted in connection with agenerated query to obtain user feedback, may be considered pendingapproval until such time as the unique response has been approved asbeing valid for the word or phrase associated with the generated query,or rejected as being invalid for the word or phrase associated with thegenerated query. As used herein, a given “unique response,” may includea set of responses similar but not exactly identical in syntax (e.g.,different punctuation or spacing); a given unique response may bereceived by way of two or more response methods. A unique response isalso referred to herein as a “unique query response” and a “unique userfeedback response.” Depending on the embodiment, more than one uniquequery response may be approved for a given generated query. Forinstance, more than one unique response may be approved as defining agiven word or phrase in connection with user feedback that being sought.Responses may, for example, be manually validated by an administrator,or the like, who reviews and dispositions the responses (e.g., possiblyonly the unique responses). Additionally, or alternatively, responsesmay be validated by an automatic process, which may approve and/orreject submitted responses based on their count and/or the thresholdsassociated with response counts. In some embodiments, an automaticvalidation process may filter the top submitted responses for a givenword or phrase, and provide those top submitted responses to anadministrator, or the like, for review and disposition. The validationphase may be performed by the response evaluation module 1324periodically (e.g., based on a schedule) or based on a condition (e.g.,where the number of submitted responses pending approval has met orexceeded a threshold).

For some embodiments, where a plurality unique user feedback responses(e.g., definitions) are provided for a given word or phrase, theresponse evaluation module 1324 may automatically (and/or through manualadmin input) approve the top ranked unique user feedback response (e.g.,most popular response) from the plurality, but may only do so if thattop ranked unique user feedback response is statistically significant.For instance, where a first unique user feedback response was receivedtwenty-six times for a given word, and a second unique user feedbackresponse was received twenty-four times for the same given word, thefirst unique user feedback response may not be approved as the responsefor the given word, even if it is the top ranked unique user feedbackresponse for the given word. This is because twenty-six is notstatistically significant over twenty-four. The first unique userfeedback may not be selected until, for example, the first unique userfeedback response remains the top ranked unique user feedback responseand the response count reaches forty.

Given that phrases may be not be identical but may be similar in natureand convey the same intent, for some embodiments, a word error rate(WER) may be used to group unique user feedback responses that comprisea phrase. For two phrases, WER may measure the substitutions, deletions,and insertions of words to convey similarity between the phrases.

For various embodiments, where a plurality unique user feedbackresponses (e.g., definitions) are provided for a given phrase, theresponse evaluation module 1324 may automatically (and/or through manualadmin input) approve the top ranked unique user feedback response.

During a check phase, the response evaluation module 1324 may determinewhether a response submitted by an audience member has beendispositioned as approved, rejected, or pending review (e.g., pendingapproval). For some embodiments, a data store may maintain the status ofwhether a unique response submitted by audience members, in connectionwith a given word or phrase, has been approved or rejected as a validdefinition for the given word or phrase. Accordingly, the check phasemay determine the disposition of a response submitted for a given wordor phrase by consulting with the data store that maintains thedisposition status of unique responses previously submitted for thegiven words or phrase; the submitted response shares the dispositionstatus of the unique response that corresponds with the submittedresponse. Depending on the embodiment, the check phase for a submittedresponse may be performed immediately or soon after the response hasbeen submitted. Where a submitted response is determined to be stillpending review during a check phase, the check phase may be re-performedat a later time, possibly following a validation phase that causes thestatus of the identical or similar submitted responses to be affected.The status of the submitted response may be updated according to thecurrent disposition of the submitted response as determined during thecheck phase. As described herein, the current status of one or moreresponses submitted by a given audience member may be provided as alisting that reflects the current statuses for those responses. Moreregarding response status is discussed later with respect to FIG. 28.

The response evaluation module 1324 evaluation of the response maycomprise determining whether the response is approved. The response maybe approved based on at least one previous response provided by anotherperson in response to another query, the other query being previouslygenerated to obtain feedback for the word or phrase from the otherperson. The response may be approved once the response is determined toaccurately define the word or phrase.

The translation data store update module 1326 may be configured toupdate a transformation or a translation (e.g., stored in thetranslation data store 210), possibly based on the evaluation of aresponse submitted, by a selected audience member, for a user feedbackquery. For example, where a submitted response in a first language isdetermined, during response evaluation, as being an approved definitionfor a given word in the (same) first language, a transformation mappingthe given word in the first language to the response in the firstlanguage will be added or updated accordingly. In another example, wherea submitted response in a second language is determined, during responseevaluation, as being an approved definition for a given word in thefirst language, a transformation mapping the given word in the firstlanguage to the response in the second language will be added or updatedaccordingly. The update of transformations or translations may beperformed by the translation data store update module 1326 during orsubsequent to a check phase that results in the disposition of asubmitted response changing to approved.

The audience competence evaluation module 1328 may be configured todetermine the competence of an audience member, which may be indicativeof the level of confidence associated with the audience member abilityto provide accurate and/or useful user feedback responses. As describedherein, the competency of a given audience member may be determinedbased on one or more feedback responses previously provided by the givenaudience member (e.g., in connection with previous queries generated andprovided to the given audience member) and/or one or more languagecapabilities of the audience member. An audience member, for example,may be evaluated to have requisite competency to provide user feedbackfor one or more words or phrases associated with a translation failurewhen the audience member has achieved a specific number of approvedfeedback responses. In another example, an audience member may beevaluated to have competency in a specific language based on at leastpreviously submitted feedback responses for a specific language, thepreferences of the audience member, and/or information related to theaudience member that indicates their fluency in the specific language.In a further example, an audience member who has previously submittedfeedback responses that were evaluated as being incorrect or fraudulent(e.g., gibberish responses) may be determined to have lower competency.

The incentive reward module 1330 may be configured to reward an audiencemember with an incentive based on the evaluation of a responsesubmitted, by the audience member, in connection with a query for userfeedback. As described herein, upon approval a submitted response, anaudience member may be rewarded with an incentive. The amount or type ofincentive rewarded may be determined based on a number of factorsincluding, without limitation, the value of the query or query responseassigned by the QR valuation module 1320, the response method used bythe audience member in responding to the query, the amount(s) ofincentives already rewarded (e.g., to the audience member or to allaudience members in connection with the particular word or phrase orthrough incentive-based user feedback), the language of the query or thelanguage of the response provided, and the type of word or phrase forwhich a response was submitted (e.g., chatspeak, abbreviation or specialdomain word or phrase). The incentive rewarded may comprise real worldcurrency or virtual currency, such as in-game currency or in-game item,which may or may not have value outside its related virtual economy(e.g., monetary value in a real world economy). For some embodiments,the incentive may comprise a real world good or service or a virtualgood or service, which may have an associated monetary value. Thoseskilled in the art recognize that other forms of incentives may berewarded in different embodiments.

For some embodiments, the incentive reward module 1330 may beresponsible for notifying an audience member when one or more of theirsubmitted responses are approved and/or when an incentive has beenawarded to the audience member for a submitted response that has beenapproved. In various embodiments, the incentive reward module 1330 maynotify the audience member of the incentive reward through anotification message (e.g., in-chat message, such as a pop-up message)and/or through an update to a listing of statuses for submittedresponses.

Those skilled in the art will appreciate that for various embodiments, asystem for user feedback may include more or less components than thoseillustrated in FIG. 13, and each component illustrated in FIG. 13 mayperform more or less operations than those described for each component.

FIG. 14 is a block diagram illustrating an example user feedback clientsystem in accordance with various embodiments. For illustrativepurposes, in FIG. 14 the user feedback client system is shown as part ofa chat client system 1400 in accordance with various embodiments. Forexample, the example user feedback client system may be part of a gamechat client system available in conjunction with an MMO game (e.g.,in-game chat client system), where various players of the MMO game canchoose to provide user feedback for flagged words or phrases, possiblyin return for in-game currency/credit/item as an incentive for thefeedback. Those skilled in the art will appreciate that for someembodiments, the example system user feedback, some or all of itsrelated components, may be separate from the example communicationtransformation and translation system 1300.

As shown in FIG. 14, the chat client system 1400 may comprise a chatclient controller 1402, a chat client communications module 1404, and achat client graphical user interface (GUI) module 1406. In someembodiments, the chat client controller 1402, the chat clientcommunications module 1404, and the chat client GUI module 1406 may besimilar to the components of the chat client system 104 as describedherein.

As also shown in FIG. 14, the chat client system 1400 may comprise atransformation/user feedback (UF) query preferences module 1408 and atransformation/user feedback (UF) query graphical user interface (GUI)module 1410. For some embodiments, the UF query preferences module 1408and/or the UF query GUI module 1410 facilitate user feedbackinteractions with respect to the CTT system 1300. In the context of thechat client system 1400, a chat user of the chat client system 1400 canbe an audience member with respect to the user feedback systems ofvarious embodiments (e.g., the CTT system 1300).

The UF query preferences module 1408 may be configured to manage andotherwise permit a chat user to preview, defined, and/or adjustpreferences in relation to the user feedback features provided inconnection with user feedback systems of some embodiment (e.g., the CTTsystem 1300). Example of preferences managed by UF query preferencesmodule 1408 may include, for instance, language preferences relating touser feedback (e.g., language of words or phrases solicited for userfeedback and/or language of the user feedback sought), preferredresponse methods for user feedback queries (e.g., select-form responsesover free-form responses), or preferred word or phrase types (e.g.,abbreviations, chatspeak, physics related, or idioms), and the like.

As used herein, a select-form response is a response that ispredetermined and selectable from a listing of two or more select-formresponses. Depending on the embodiment, a listing of select-formresponses may permit an audience member to select two or more responseswhen applicable. A free-form response is a response that comprises atext-based value (e.g., character value or string value) entered into afield by an audience member.

The UF query GUI module 1410 may graphically facilitate the presentationof a query generated for user feedback and provided to a chat user(e.g., by the CTT system 1300), presentation of one or more responsemethods associated with the query, and/or receiving a response from thechat user through the presented response method. The UF query GUI module1410 may also facilitate management of management of preferences throughthe UF query preferences module 1408. More regarding with the graphicaluser interfaces that may be presented at a chat client system isdescribed later with respect to FIGS. 17-23 and 24-31.

Those skilled in the art will appreciate that for various embodiments, aclient system for user feedback may include more or less components thanthose illustrated in FIG. 14, and each component illustrated in FIG. 14may perform more or less operations than those described for eachcomponent.

FIG. 15 is a flowchart illustrating an example method 1500 for userfeedback in accordance with various embodiments. At step 1502, thetranslation failure management module 1314 may identify a potentialfailure of a transformation or translation of a text, possibly from afirst language to a second language. At step 1504, the translationfailure management module 1314 may also select a word or phrase, fromthe identified potential failure, for user feedback. At step 1506, theUF query audience selection module 1318 may select an audience memberfor soliciting user feedback. At step 1508, the UF query generationmodule 1316 may generate a query to obtain the user feedback, possiblyfrom the selected audience member. At step 1510, the response evaluationmodule 1324 may receive a response to the generated query. The query APImodule 1322 may be responsible for providing the generated query to theselected audience member, and receiving the response to the generatedquery. At step 1512, the response evaluation module 1324 may evaluatedthe received the response. At step 1514, the audience competenceevaluation module 1328 may evaluate the competence of the selectedaudience member, possibly based on the response provided in step 1510and/or the evaluation of the received response as performed in step1512. At step 1516, the incentive reward module 1330 may be reward theselected audience member an incentive based on the response evaluation.As noted herein, upon evaluating a response and determining that it isapproved, the incentive reward module 1330 may reward the audiencemember with a reward, possibly in accordance with the value of the queryand/or the query response as determined by the QR valuation module 1320.At step 1518, the translation data store update module 1326 may update atransformation or translation based on the response evaluation. As notedherein, upon evaluating a response and determine that it is approved,the translation data store update module 1326 may update a translationor transformation that correspond to the word or phrase of the query andthe submitted query response.

FIG. 16 is a block diagram illustrating an example data flow 1600 for auser feedback system in accordance with various embodiments. As shown,the data flow 1600 involve a chat client system 1400, a translationfailure management module 1314, the UF query generation module 1316, theUF query audience selection module 1318, the response evaluation module1324, the translation data store update module 1326, and the incentivereward module 1330. The data flow 1600 further involve an undefinedwords/phrases data store 1602, a recorded responses data store 1604, afeedback audience data store 1606, a response approval data store 1608,a chat data store 1610, and a dictionary data store 1612.

The undefined words/phrases data store 1602 may comprise a word orphrase associated with a translation failure and for which user feedbackis being sought. The undefined words/phrases data store 1602 mayinclude, with the word or phrase, a sample sentence in which the word orphrase is used (e.g., word or phrase context), a confidence measure thatindicates how important the word or phrase is (e.g., word importance),source language for the word or phrase, a target language for the userfeedback sought, and the like. In some embodiments, the word importanceof a word or phrase in the undefined words/phrases data store 1602 mayinitial equal for all words but gets increased as the word or phrase isencountered and problematic and/or untranslatable.

The recorded responses data store 1604 may comprise a user feedbackresponse, received from an audience member and recorded for a word orphrase included in the undefined words/phrases data store 1602. In someembodiments, the user feedback response comprises a response receivedfor a query generated to obtain user feedback with respect to the wordor phrase. The undefined words/phrases data store 1602 may include, withthe recorded user feedback response, an identifier for the audiencemember submitting the user feedback response, a timestamp for when theuser feedback response was received and/or recorded, an indication ofwhether the recorded user feedback response is approved, a timestamp forwhen the recorded user feedback response is approved, and the like.

The feedback audience data store 1606 may comprise a set of identifiersfor audience members that chosen to participate in user feedback for aword or phrase included in the undefined words/phrases data store 1602.The feedback audience data store 1606 may include, with each identifierfor an audience member, a confidence score that reflects theconsistency, competency, and/or confidence of the audience member inproviding user feedback responses.

The response approval data store 1608 may comprise each unique userfeedback response received in connection with a word or phrase includedin the undefined words/phrases data store 1602. The response approvaldata store 1608 may include, with each unique user feedback response, anindication of whether the unique user feedback response is an approvedresponse (e.g., correct response), a rejected response (e.g., incorrectresponse), a response pending review (e.g., response needing review), ora response having some other status. In some embodiments, the responseapproval data store 1608 may be employed in determining when a userfeedback response received from an audience member and recorded in therecorded responses data store 1604 has been approved.

According to some embodiments, the translation failure management module1314 may be configured to review chat logs, possibly provided by thechat data store 1610, and identify one or more words or phrasesassociated with actual or potential translation failures. In variousembodiments, the translation failure management module 1314 may beconfigured to exclude those words, or phrases, defined in the dictionarydata store 1612, which may comprise a standard dictionary (e.g., Oxforddictionary) and/or a dictionary of words or phrases (e.g., chatspeakwords or phrases) that an embodiment described herein can parse,recognize, and/or handle. Words or phrases identified by the translationfailure management module 1314 may be added to the undefinedwords/phrases data store 1602, thereby enabling those added words andphrases to be selected for user feedback from select audience members.

The translation failure management module 1314 may be configured toselect one or more words or phrases, possibly from the undefinedwords/phrases data store 1602, for user feedback. For some embodiments,the translation failure management module 1314 may select from a set ofwords or phrases designated as having the highest importance in theundefined words/phrases data store 1602 (e.g., selected from top 10important words or phrases in the undefined words/phrases data store1602). In some embodiments, the translation failure management module1314 may select two or more words or phrases so that an audience membercan be provided with a set of two or more user feedbacks from which tochoose to respond (e.g., enable the audience member to choose those userfeedbacks to which they feel most confident responding). The selectionprocess by the translation failure management module 1314 from theundefined words/phrases data store 1602 may be random, based on word orphrase importance, age of the word or phrase in the undefinedwords/phrases data store 1602, a preference of the selected audiencemember by the UF query audience selection module 1318, whether theaudience member selected by the UF query audience selection module 1318has already responded to the word or phrase to be selected (e.g.,determine based on checking the recorded responses data store 1604 forthe word or phrase to be selected), and the like.

The UF query audience selection module 1318 may be configured to selectone or more audience members, possibly from the feedback audience datastore 1606, from whom user feedback may be sought. As described herein,the user feedback may be sought for the words or phrases selected by thetranslation failure management module 1314, possibly from the chat datastore 1610. The selection of an audience member from the feedbackaudience data store 1606 may be dependent on the competency levelassociated with the audience member.

The UF query generation module 1316 may be configured to generate one ormore queries for the words or phrases selected by the translationfailure management module 1314, possibly from the undefinedwords/phrases data store 1602, for user feedback. As shown, thetranslation failure management module 1314 may provide the UF querygeneration module 1316 with the selected words or phrases for which oneor more queries are to be generated. As described herein, the UF querygeneration module 1316 may consider a number of different factors whengenerating the query including, for instance, the preferences of theaudience members selected by the UF query audience selection module 1318and the word or phrase selected for user feedback by the translationfailure management module 1314. Eventually, the UF query generationmodule 1316 may provide the chat client system 1400 with the one or morequeries generated by the UF query generation module 1316, which may havegenerated a different query for each word selected and provided by thetranslation failure management module 1314.

Eventually, the one or more queries generated by the UF query generationmodule 1316 may be provided to the chat client system 1400, which inturn would present the provided queries for selection by a user at thechat client system 1400. Depending on the embodiment, the UF querygeneration module 1316 may provide the generated queries to the chatclient system or, alternatively, another component may be responsiblefor providing the generated queries to the chat client system. Oncepresented to the generated queries are presented for selection at theclient chat system 1400, the user at the client chat system 1400 maychoose to respond to one or more of the presented queries, and thoseresponses provided by the chat client system 1400 may be added (e.g.,recorded) to the recorded responses data store 1604.

When a response is added to the recorded responses data store 1604, someembodiments check the added response may be evaluated by the responseevaluation module 1324. As described herein, the response evaluationmodule 1324 may evaluate of a response by check the response anddisposition the status of a response.

As shown in FIG. 16, the response evaluation module 1324 comprises aresponse check module 1614, which may be configured to perform a statuscheck of a user feedback response during evaluation of the user feedbackresponse. The response check module 1614 may check the status of a userfeedback response from the client chat system 1400 by retrieving theuser feedback response from the recorded responses data store 1604 andchecking the status of the unique response in the response approval datastore 1608 that corresponds to the retrieved user feedback response. Indoing so, the response check module 1614 can determine whether a givenuser feedback response is approved or rejected. The approval status ofthe retrieved user feedback response in the recorded responses datastore 1604 may be updated according to the latest status check performedby the response check module 1614. Where the response check module 1614determines that a retrieved user feedback response has been approved,the approval status of the retrieved user feedback response in therecorded responses data store 1604 may be updated to reflect theapproval and to include a timestamp for when the approval status wasupdated. Eventually, the approval reflected in the recorded responsesdata store 1604 for the retrieved user feedback response may result inthe incentive reward module 1330 rewarding an incentive to the audiencemember that submitted the approved user feedback response.

If the status of the retrieved translation response is still pendingreview, the response check module 1614 may re-check the status of theretrieved user feedback response at a later time (e.g., according to apredetermined schedule). If the status of the retrieved translationresponse is rejection, the approval status of the retrieved userfeedback response in the recorded responses data store 1604 may beupdated to reflect the rejection.

Where a unique response corresponding to the retrieved user feedbackresponse is not found, the retrieved user feedback response can be addedto the response approval data store 1608 as a unique response for theword or phrase for which the user feedback response was provided (e.g.,by a user at the chat client system 1400). Additionally, where aretrieved user feedback response is added to the response approval datastore 1608 as a unique response, the unique response may have theinitial status of pending approval, which will remain until such time asthe status of the unique response is manually or automaticallydispositioned (e.g., through the evaluation response module 1324).

As shown in FIG. 16, the response evaluation module 1324 also comprisesa response validator 1616, which may be configured to disposition thestatus a unique user feedback response as being approved, rejected, orpending approval. As described herein, a unique user feedback response,submitted in connection with a generated query to obtain user feedback,may be considered to be pending approval until such time as the uniqueuser feedback response has been approved as being valid for the word, orphrase associated with the generated query or rejected as being invalidfor the word or phrase associated with the generated query.

For some embodiments, the response evaluation module 1324 may beconfigured to determine when a given word or phrase no longer in need ofadditional user feedback. The response evaluation module 1324 may makesuch a determination based on such examples of factors as how manyunique user feedback response have been approved for the given word orphrase, and whether the count of a unique and approved user feedbackresponse has met or exceeded a specific threshold. When a given word orphrase is determined to longer need further user feedback, the responseevaluation module 1324 may be configured to remove the given word orphrase from the undefined words/phrases data store 1602, therebyremoving the word or phrase from future selection (e.g., by thetranslation failure management module 1314) for user feedback.

As described herein, the incentive reward module 1330 may reward anincentive to an audience member once a user feedback response they haveprovided has been approved as a valid response for the word or phrasefor which the user feedback response was provided (e.g., by the audiencemember through the chat client system 1400). The incentive reward module1330 may identify one or more user feedback responses, in the recordedresponses data store 1604, that were recently approved (e.g., theirapproval status were recently updated to reflect the approval) and/orthat were approved since the last time the incentive reward module 1330attempted to identify one or more user feedback responses in therecorded responses data store 1604 having an approved status. Theincentive reward module 1330 may determine when a given user feedbackresponse was last approved based on the approval timestamp included forthe user feedback response in the recorded responses data store 1604.For some embodiments, once an incentive is rewarded for a translatedfeedback response in the recorded responses data store 1604, thetranslated feedback response may be removed from the recorded responsesdata store 1604. Alternatively, once an incentive is rewarded for atranslated feedback response in the recorded responses data store 1604,the translated feedback response may be updated in the recordedresponses data store 1604 to indicate, for instance, when an incentivehas been rewarded, the amount of incentive rewarded, the type ofincentive rewarded, when the audience member was notified of the reward,and/or how the audience member was notified of the incentive reward.

FIG. 17 depicts example screenshots for receiving user feedback for aword in accordance with various embodiments. In particular, FIG. 17presents screenshots 1702, 1704, and 1706, which represent examples ofGUIs that may be presented to an audience member (e.g., through the chatclient system 1400) to facilitate user feedback processes. Thescreenshot 1702 presents an example of a banner 1708 that solicits oneor more audience members to participate in a user feedback for a word orphrase associated with a translation failure. An audience member maychoose to participate in user feedback by selecting the banner 1708,which may lead in the commencement of a user feedback session and/orlead the audience member to a listing of available user feedbackopportunities from which the audience member can choose to participate.As described herein, a user feedback opportunity may permit an audiencemember to provide a definition for a word or phrase associated with anactual or potential translation failure. In accordance with someembodiments, the audience member can select one of the available userfeedback opportunities associated with a word or phrase and, then,provide a definition for the associated word or phrase when prompted todo so.

The screenshot 1704 presents a listing 1710 of available user feedbackopportunities for various words (e.g., “Skrilla,” “Booty,” “Cray,”“Hecka,” and “Freshness”). The screenshot 1706 provides an example of aquery 1712 presented to an audience member to obtain user feedback forthe word “Skrilla.” As shown, the query 1712 provides an example context1714 in which the word “Skrilla” is used, and also provides a field 1716configured to receive a free-form response for the query 1712. Anaudience member may be led to the screenshot 1706 when the user feedbackfor the word “Skrilla” is selected by the audience member from thelisting 1710 of screenshot 1704.

FIG. 18 depicts example screenshots for skipping user feedback inaccordance with various embodiments. In particular, FIG. 18 presentsscreenshots 1802 and 1804, which represent examples of GUIs that may bepresented to an audience member (e.g., through the chat client system1400) to facilitate user feedback processes. The screenshot 1802presents a listing 1806 of user feedback opportunities available forselection by an audience member. As shown, the listing 1806 provides anaudience member with the option to skip one or more of the user feedbackopportunities listed.

The screenshot 1804 presents an example of a query 1808 presented to anaudience member to obtain user feedback for the various words. As shown,the query 1808 provides an audience member with the option to skip theprocess of providing a response to the query 1808. Certain embodimentsmay avoid inaccurate and/or fabricated responses to various userfeedback queries by providing an audience member with the option to skipcertain user feedback opportunities and/or various user feedbackqueries,

FIG. 19 depicts example screenshots for receiving user feedback for aphrase in accordance with various embodiments. In particular, FIG. 19presents screenshots 1902 and 1904, which represent examples of GUIsthat may be presented to an audience member (e.g., through the chatclient system 1400) to facilitate user feedback processes. Thescreenshot 1902 presents a listing 1906 of user feedback opportunitiesavailable for selection by an audience member. As shown, the listing1906 of available user feedback opportunities for various words andphrases (e.g., “Skrilla,” and “Pardon my french”).

The screenshot 1904 provides an example of a query 1908 presented to anaudience member to obtain user feedback for the phrase “Pardon myfrench.” As shown, the query 1908 provides an example context 1910 inwhich the phrase “Pardon my french” is used, and also provides a field1912 configured to receive a free-form response for the query 1910. Anaudience member may be led to the screenshot 1904 when the user feedbackfor the phrase “Pardon my french” is selected by the audience memberfrom the listing 1906 of screenshot 1902.

FIG. 20 depicts example screenshots for receiving user feedback througha listing of select-form responses in accordance with variousembodiments. In particular, FIG. 20 presents screenshots 2002, 2004, and2006, which represent examples of GUIs that may be presented to anaudience member (e.g., through the chat client system 1400) tofacilitate user feedback processes. The screenshot 2002 presents anexample of a banner 2008 that solicits one or more audience members toparticipate in a user feedback for a word or phrase associated with atranslation failure. An audience member may choose to participate inuser feedback by selecting the banner 2008, which may lead in thecommencement of a user feedback session and/or lead the audience memberto a listing of available user feedback opportunities from which theaudience member can choose to participate.

The screenshot 2004 provides an example of a query 2010 presented to anaudience member to obtain user feedback for the word “Skrilla.” Includedwith the query 2010 is a select-form responses 2012, which listspossible responses from which an audience member can select. Thescreenshot 2006 presents an incentive (e.g., 5 gold coins) beingrewarded by notifications 2014 and 2016 once a correct response “money”is selected for the word “Skrilla.”

FIG. 21 depicts example screenshots for creating a listing of selectionsin accordance with various embodiments. In FIG. 21, a screenshot 2102provides an example of a query 2106 presented to an audience member toobtain user feedback for the word “Skrilla.” As shown, the query 2106provides an example context 2108 in which the word “Skrilla” is used,and also provides a field 2110 configured to receive a free-formresponse for the query 2106.

According to some embodiments, a select-form response method, used toobtain user feedback for a given word or phrase, may comprise a listingof predefined responses selected from free-form responses gathered forthe given word or phrase. Accordingly, as various audience membersprovide free-form responses for the word “Skrilla” through the field2110 (e.g., “A lot of money,” “Cash,” “Money,” and “Really Rich”), theresponse collected may be useful in creating a listing of select-formresponses 2112, as shown in the screenshot 2104.

FIG. 22 depicts screenshots illustrating example incentive notificationsin accordance with various embodiments. In FIG. 22, a screenshot 2200presents an example of a notification to an audience member notifyingthem of the approval of their response of “money” for the word“Skrilla,” and notify them of an incentive rewarded for the approvedresponse (e.g., XXXX Gold). A screenshot 2202 presents an example of anotification to an audience member notifying them of the rejection oftheir response of “money” for the word “Skrilla.” The screenshot 2204presents an example of a push notification to an audience membernotifying them of the approval of their response.

FIG. 23 depicts screenshots illustrating an example of when atranslation has failed between client chat systems in accordance withvarious embodiments. In FIG. 23, a screenshot 2300 represents an exampleinterface of a first chat client system and a screenshot 2302representing an example interface of a second chat client system. Adouble arrow 2304 represents chat communications between the first andsecond chat client systems. As shown, as chat user “Aramis” enters chatcommunications into the interface of the first chat client system inEnglish, the entered chat communications is translated to French andpresented on the interface of the second chat client system of chat user“tapir.” Likewise, as chat user “tapir” enters chat communications intothe interface of the second chat client system in French, the enteredchat communications is translated to English and presented on theinterface of the first chat client system of chat user “Aramis.”

As shown in FIG. 23, chat communication 2306 (i.e., “Tru dat bro?”)entered by chat user “Aramis” in the interface of the first chat clientsystem fails to translate when it is sent to the interface of the secondchat client system of chat user “tapir.” The chat communication 2308(i.e., “Tru dat bro?”) presented to chat user “tapir” reflects thistranslation failure, by presenting the original chat communicationentered by chat user “Aramis” and indicating to chat user “tapir” thatthe chat communication is the original chat message entered by chat user“Aramis.”

The translation failure illustrated by FIG. 23 may be one that canbenefit from user feedback in accordance with some embodiments. Inaccordance with some embodiments, the translation failure illustrated inFIG. 23 may be identified by the translation failure management module1314 and one or more words from the original chat communication 2306(i.e., “Tru dat bro?”) may be added to the undefined words/phrases datastore 1602 for future selection for user feedback from participatingaudience members. For example, each of words “Tru,” “dat,” and “bro” maybe added to the undefined words/phrases data store 1602 for future userfeedback of each.

FIGS. 24 and 25 depict screenshots illustrating example listings ofwords or phrases available for user feedback in accordance with variousembodiments. In FIG. 24, a screenshot 2400 presents a listing 2402 ofuser feedback opportunities, available for audience member selection,for words and phrases (including “Tru”), which are available forselection by an audience member. In FIG. 25, a screenshot 2500 presentsanother listing 2502 of user feedback opportunities, available foraudience member selection, for phrases available for selection by anaudience member. In both FIGS. 24 and 25, the screenshots 2400 and 2500may be part of an in-game chat system, whereby game players can provideuser feedback for certain words or phrases and, upon approval of thefeedback response, in-game credit (e.g., in-game gold) may be awarded.

FIG. 26 depicts a screenshot illustrating an example of defining a wordin accordance with various embodiments. In FIG. 26, the screenshot 2600presents a query 2602 that includes an example context 2604 in which theword “Tru” is used, and also provides a field 2606 configured to receivea free-form response for the query 2602.

FIG. 27 depicts a screenshot illustrating an example listing ofselect-form responses in accordance with various embodiments. In FIG.27, the screenshot 2700 presents a listing 2702 of responses that anaudience member can select to define the word “nemore.”

FIG. 28 depicts a screenshot illustrating an example listing of statusesfor responses submitted in accordance with various embodiments. As shownin FIG. 28, a listing 2802 of submitted response statuses includes apending status for a first response 2804, and approved statuses for thesecond and third responses 2806 and 2808. For some embodiments, the list2802 may provide further information for response statuses including,for instance, why a particular response has been approved, rejected, orstill pending review.

FIG. 29 depicts a screenshot illustrating an example incentivenotification in accordance with various embodiments. In particular, FIG.29 provides a screenshot 2900 that presents an example notification 2902to a member for correctly defining the phrase “U still thr” as “Youstill there?” during a user feedback process in accordance with anembodiment. The notification indicates that as an incentive for the userfeedback provided, he or she will be rewarded with 10 gold pieces, whichmay be of value or useful in as in-game currency. As shown, thenotification also provides a summary of the user feedback (i.e., theword or phrase in question and the user feedback response provided).

In certain embodiments, a learning system for data selection is providedin which feedback obtained from users is automated by a machine learningsystem that has checks and balances for player consistency. The systemadds parallel sentences received from players to parallel corpora whichcan be used to retrain the statistical machine translation (SMT) systemsfrom time to time.

A chat transformation system may be or include a system that transformschatspeak to plain speak. For example, the chat transformation maytransform “U r da king” (a chatspeak message) to “You are the king” (aplain speak message). In certain embodiments, “plain speak” refers toordinary language spoken and/or written by ordinary individuals, usuallyoutside of the electronic chat environment where chatspeak maypredominate. Plain speak tends to be more grammatical than chatspeak.

The learning system may also utilize or include a language translationsystem that translates one language to another. For example, thelanguage translation may translate “How are you doing kind sir” (anEnglish message) to “

Como to va amable señor” (a Spanish message).

In some embodiments. “parallel corpora” is understood to mean two texts,one in each language, aligned in parallel such that line n in one textcorresponds to line n in the second translated text. Parallel corporamay also be referred to as “training corpora” in such contexts.

In various embodiments, “machine learning” is understood to refer to asupervised, semi-supervised or unsupervised system that can learn frompatterns in input data and develop mechanisms to detect, transform, orpredict behavior.

In general, building chat transformation systems and languagetranslation systems requires a moderate amount of syntactic rules or alarge amount of parallel corpora for statistical learning. The systemsand methods described herein generally utilize reliable statistical chattransformation and language translation systems that use parallelcorpora. In certain situations, however, this initial training data setmay be limited in its content and scope. For example, new chat words arecreated and added to chat rooms each day. To maintain accurate andreliable transformation and translation systems, these new chat wordsshould be augmented into the chat transformation training corpora.

In various embodiments, systems and methods are provided for identifyingwords that are “Out of Vocabulary” (OOV) (e.g., words that are notpresent in a given lexicon). Referring to FIG. 30, in some embodiments,a method 3000 is provided for detecting and processing OOV words. Atstep 3002, the OVV words are initially detected by sending them througha translator system, such as the CTT system 114 or the CTT system 1300and/or one or more modules thereof. When the output from the translatorsystem is same as the input for a given word, the translator systemindicates a lack of transformability, which suggests the word may beOOV. To further evaluate the word as a potential OOV word, the systemsand methods may determine (step 3004) whether the word is a new word asopposed to just a misspelled word, both of which will appear as OOV.Accordingly, words that can be corrected with a spell checker may beconsidered to be misspelled words, rather than OOV words.

Additionally, OOV words that frequently appear in chats generally have ahigher propensity of being an OOV word (e.g., a new chat speak word).For example, when a word has been used by users in prior text messages,such prior use suggests the word is likely an OOV word. In someembodiments, an ensemble of machine learning and language processingmethods is used in parallel to detect whether a word is an OOV word(step 3006).

Additionally or alternatively, Bayesian probabilities may be computed(step 3008) to provide a statistical probability of when an OOV is a newword, rather than a misspelled word. A genuine chatspeak word tends tofollow certain words commonly used prior and post the chatspeak word. Aspelling error in comparison will have a less consistent distribution ofneighboring words. Computing the prior and posterior Bayesianprobabilities will help distinguish useful OOV words which could beadded to a lexicon, from spelling errors which should not be added to alexicon. For example, consider the phrase “Wassup, how's it going.”“Wassup” is considered an OOV word as it is not present in standardlexicon. But “Wassup” is almost always followed by the words “How's itgoing” or is often used at the beginning of the sentence. This patternor consistent behavior is captured by Bayesian probabilities. The systemmay be trained on texts that have misspelled words but no chat words.

Alternatively or additionally, machine learning methods such as k-meansclustering may be used to distinguish (step 3010) between differentkinds of OOV words, such as new chat words, misspelled words, or junk.K-means clustering tends to bring out latent similarities betweenclasses of words. Words belonging to a similar topic tend to beclustered together indicating a latent synonymous relationship betweenthem. Consider the example of “Wassup, how's it going” again. Clusteringa group of sentences using the k-means algorithm, reveals a cluster ofgreeting words such as “Hi,” “What's up,” “Hello,” “Hi!,” etc., with“Wassup” included among them, within the cluster. A spelling error, bycontrast, would be placed in the fringes of clusters or not in anydefined cluster at all. These latent relationships help distinguishuseful OOV words from errors. The syntax and semantics of a sentence maybe analyzed to determine what kind of OOV word the sentence includes(e.g., a verb, noun, or adjective).

When the systems and methods detect a new chat word or other OOV word,the new chat word may be presented to a human translator to define achat transformed or language translated version of the new chat word.The transformed or translated version of the new chat word may then beadded to the translation lexicon and used by the systems and methodsdescribed herein.

As described herein, when an incentive is provided for manualtranslation of chats between languages, there is potential for users tofraudulently manipulate the system to take advantage of the incentive(e.g., in-game currency). The systems and methods described herein aregenerally tolerant to human-translator abilities yet able to detectfraudulent submissions.

When a user of the system acts as a translator, the user translates oneor more words or sentences into the target language specified. The usercommits fraud, however, when the user gives a false, incomplete, orimproper translation for the sake of gaming the system or for gainingincentive without fulfilling the purpose of the system.

Referring to FIG. 31A, in certain embodiments, the systems and methodsdescribed herein utilize a fraud detection module 3100. The frauddetection module 3100 detects fraud in incentivized translations bypresenting users with both new and old training data (e.g., parallelcorpora). Old training data corresponds to translations for which thecorrect answers are known, while new training data corresponds totranslations for which the correct answers are not known. The percentageof new to old data may be varied to a user over time. For example, moreold data may be presented initially and then decreased in percentagegradually.

In some embodiments, fraud detection is done by checking the accuracy ofold data translations received from users. A confidence score isassigned to each user based on this accuracy. Large or sudden shifts intranslation accuracies or consistently low accuracies are indicative offraud or low translation capabilities in a user. Even after establishingconfidence in the capabilities of a translator, old data is preferablyseeded randomly, at least 10-20% of the time for periodic fraudchecking.

Using this basic structure, the fraud detection module 3100 may includeand/or utilize a supervised fraud detection module 3102 and/or anunsupervised fraud detection module 3104. With the supervised frauddetection module 3102, a reporting tool may present the output from eachuser in, for example, a user interface with the following fields: inputsentence presented, translation obtained, existing known to be truetranslation, current confidence score of the user, and a graph showingthe variation of the translator's confidence score over time. Whenreviewing a translation, a human supervisor may accept or reject thetranslation, and may adjust the user's confidence score accordingly. Thesupervisor may remove the user (i.e., revoke the user's translationprivileges) if cumulative reports show fraudulent behavior. Removal ofthe user or revocation of the user's translation privileges may beperformed using a translation privileges module 3106.

Alternatively or additionally, the fraud detection module 3100 mayutilize the unsupervised fraud detection module 3104. With theunsupervised fraud detection module 3104, the accuracy of translationsmay be computed using various metrics, such as WER (word error rate) andBLEU (machine translation accuracy metric that compares machinetranslations with good quality reference translations). Confidence inuser translation abilities may be checked for changes or variations(e.g., upswings or downswings). Similar sentences presented to onetranslator may be presented to other independent translators who use thesystem. Inter-translator reliabilities may also be computed. Forexample, collusion may be avoided between translators through randomsampling, social network analysis (e.g., to confirm two translators arenot connected socially or do not have a pre-existing relationship), andby detecting repeated interactions among users in chat sessions and/oronline gaming. Two users who regularly interact together online (e.g.,in an online game or chat session) may be more likely to engage incollusion. In some embodiments, item response theory (i.e., a theoryused in psycholinguistics and testing theory) is used to augmentmeasurement of translator confidence with translator ability. Frauddetection may be performed using item response theory to do unsupervisedfraud detection in a translation augmentation system, which has anincentive mechanism. Item response theory dictates ways in which thetranslator accuracy can be measured relative to peers and to themselvesover a period of time, to measure consistency. Deviations from the normmay be identified with this method. Intra-translator reliabilities mayalso be computed by presenting the same sentence to a translator againafter a set period of time. Various thresholds in reliabilities andtranslator confidences may be set and, if a translator's confidencefalls below such thresholds, the translator may be removed and blockedfrom the system (e.g., the user's translation privileges may berevoked), using the translation privileges module 3106. In someimplementations, translations from high confidence systems are added tothe translation pair lexicons.

FIG. 31B includes a flowchart of a method 3110 of detecting fraud inincentivized translations, in accordance with certain embodiments of theinvention. The method includes selecting (step 3112) a mixture of oldtraining data and new training data. The old training data includes oneor more old text messages for which correct translations to a differentlanguage are known. The new training data includes or more new textmessages for which correct translations to the different language arenot known. A plurality of respective requests are sent (step 3114) atdifferent times to a client device of a user. The requests include (i) arequest for the user to translate the old training data and/or the newtraining data and (ii) an incentive for the translation. After sending aparticular request, a translation is received (step 3116) from theclient device for the old training data of the particular request. Thereceived translation is compared (step 3118) with the correcttranslation for the old training data. An accuracy of the receivedtranslation is determined (step 3120) based on the comparison. Next, aconfidence score is updated (step 3122) for the user, based on thetranslation. The confidence score represents a likelihood that the userwill provide an accurate translation of a text message to the differentlanguage at a later time.

In various embodiments, the systems and methods described herein utilizevoice translation or voice recognition technology to translate audiblespeech in one language to another language for users of a group voicechat system. The systems and methods may be implemented for chatspeak inwhich a speech-to-text transcribing system transcribes user chatspeakinto text, this text is then transformed to plain speak (e.g.,non-chatspeak) and translated to a foreign language. A finaltransformation is then done to produce foreign chat speak which is thenoutputted to the end user through a foreign language text-to-speechsystem. The systems and methods preferably use state of the art speechrecognition techniques and statistical machine translation techniqueswith extremely fast decoders.

FIG. 32 is a schematic diagram of a group chat system 3200 that allows agroup of people 3202 who speak different languages to interact verballyusing chatspeak. As described herein, the system 3200 is able toidentify the languages spoken by the people participating in the groupchat system 3200. When a first user 3204 wishes to send an audiblechatspeak message to a second user 3206, the first user 3204 inputs anaudible chatspeak message 3208 in a first language (e.g., English) to auser input device (e.g., a microphone in a chat client system). A speechrecognition module 3210 converts the audible chatspeak message to achatspeak text message 3212 in the first language. A transformationmodule 3214 is used to transform the chatspeak text message 3212 to aplain speak (e.g., non-chatspeak) text message 3216 in the firstlanguage. Next, a translation module 3218 is used to translate the plainspeak text message 3216 to a corresponding plain speak text message 3220in the second language (e.g., French). A transformation module 3222 isthen used to transform the corresponding plain speak text message 3220to a corresponding chatspeak text message 3224 in the second language.As one of ordinary skill in the art will recognize, the transformationmodule 3222 may be the same as or form part of the transformation module3214. A text-to-speech module 3226 is then used to convert thecorresponding chatspeak text message 3224 to a corresponding chatspeakaudible message 3228 in the second language. Finally, the correspondingchatspeak audible message 3228 is delivered to the second user 3206using an output device (e.g., a speaker on a second chat client system).

In various embodiments, the speech recognition module 3210 may utilizehidden Markov models, dynamic time warping (DTW)-based speechrecognition, and/or neural networks to convert the audible chatspeaktext message 3208 to the chatspeak text message 3212. Likewise, thetext-to-speech module 3226 may use speech synthesis to convert thecorresponding chatspeak message into the corresponding chatspeak audiblemessage. The speech synthesis may utilize or include concatenativesynthesis (e.g., unit selection synthesis, diphone synthesis, and/ordomain-specific synthesis), formant synthesis, articulatory synthesis,HMM-based synthesis, and/or sinewave synthesis, as understood by thoseof ordinary skill in the art.

An important aspect of creating such a speech processing system involvescollecting speech samples from multiple accents and dialects forlanguages that may be processed. The nature of the speech data mayinclude chatspeak and plain speak formats of each language, so as tomaintain relevance of the system to the domain the system addresses. Theincentivized feedback mechanism described herein may be used totranscribe these speech samples, which may in turn be used to train thespeech recognition module 3210 and/or the text-to-speech module 3226.Domain adaptation techniques may be used to substitute data points wheresparse. This may be needed in the case of chatspeak speech samples wheredata tends to be sparse. For example, speech data collected in a gamedomain (e.g., for an online game) can be substituted with plain speakdata that is abundantly available. Domain adaptation preferably includesidentifying rules that govern minor speech variations from chat-plainspeak in a given language (e.g., rules that govern the conversion fromchatspeak to plain speak or from plain speak to chatspeak, in the givenlanguage). A plain speak sentence, which does not have speech samples inthe chatspeak equivalent, can then be converted to chatspeak using thesedomain level rules. A user feedback loop may be used to tune theacoustic model parameters (e.g., for the speech recognition module 3210and/or the text-to-speech module 3226) to a level that makes theacoustic model domain specific and hence more accurate. For example,when the speech recognition module 3210 consistently has difficulty witha particular accent, additional audible samples of various words may beprovided (e.g., by users) to the system in that accent. This will helpthe speech recognition module 3210 learn how to better recognize wordsspoken with the accent.

As mentioned, embodiments of the systems and methods described hereinare used to translate text or chat messages from a group chatenvironment into different languages. Archiving such translated chatsmay lead to a very large number of texts in different languages to bepersisted into a repository.

Referring to FIG. 33A, in certain embodiments, to reduce storagerequirements and facilitate review of chat histories by users, a chathistory module 3300 is used to translate chat histories in real-time asthe chat histories are browsed by users. The chat history module 3300includes a chat storage module 3302 (e.g., a register or other storagedevice) for storing chat histories from the various users. The chathistory module 3300 also includes a chat history transformation module3304 that transforms a text message before and/or after the text messageis translated to a different language. For example, the chat historytransformation module 3304 may perform a real-time transformation of achat history text message from chatspeak to formal speak or plain speak.In some embodiments, the chat history transformation module 3304 is thesame as or similar to the transformation module 208. The chat historymodule 3300 also includes a chat history translation module 3306, whichmay be used to perform a real-time translation of a chat history textmessage (e.g., in formal speak or plain speak) to a different language(e.g., from French to English). The chat history translation module 3306may be or include other modules or components described herein, such asthe language module 206 and/or the translation data store 210.

Once the user is done reviewing a chat history, any transformed and/ortranslated text generated by the chat history module 3300 may be deletedor removed from memory. This reduces storage requirements for thesystems and methods. If the user wishes to review the chat history at alater time, the chat history module 3300 may be used again to transformand translate the text in the chat history, as required.

In certain embodiments, the chat history module 3300 translates a chathistory for a user in real-time. The chat history module 3300 receives arequest from a user to review a history of text messages from a chatsession. The chat history module 3300 receives, from the chat historystorage module 3302, the history of text messages, which includes textmessages in a plurality of languages. The chat history transformationmodule 3304 and the chat history translation module 3306 are then usedto transform and/or translate an initial portion of the chat history, asrequired, into a language used by the user. After viewing the translatedfirst portion of the chat history, the user may wish to view a differentportion of the chat history. The chat history module 3300 may thenreceive a request from the user to view the different portion of thehistory of text messages. The chat history transformation module 3304and the chat history translation module 3306 are then used to transformand/or translate the different portion of the chat history, as required,into a language used by the user. The chat history module 3300preferably performs the transformations and/or translations inreal-time, as the user scrolls through the chat history.

In certain instances, scrolling through chat history presents a problemof scale and data storage. Offering infinite scrolling of chat history,presents a problem of fast real-time access of data spanning multipledatabases and multiple users. This may be done by spawning multipleprocesses in parallel that fetch historical messages from all userspresent in a chat room. Translation and associated chat transformationson these messages may be done in parallel, as the messages are fetchedfrom the data storage. The resultant output realized by the end user isthat of a seamless transition from one screen of chats to the next,where data lookup from the database has already been done. This can goon for an infinite number of screens, as the systems and methodsdescribed herein may have no limitations on data storage and parallelcomputation may be recycled between processes that were spawned earlier.

FIG. 33B is a flowchart of a method 3310 of translating chat historiesin real-time, in accordance with certain embodiments of the invention.The method 3310 includes receiving (step 3312) a request from a personto review a history of text messages from a chat session. The historypreferably includes text messages in a plurality of languages and from aplurality of users. At least two parallel processes are performed (step3314). Each parallel process includes (i) receiving or selecting a textmessage generated by a respective user of the chat session (i.e., thetext message forming at least part of the history of text messages), and(ii) translating the text message into a target language. Translatedtext messages from the plurality of parallel processes are provided(step 3316) to a client device of the person. A request is received(step 3318) from the person to review a different portion of the historyof text messages. Steps 3314 and 3316 are repeated for the differentportion of the history of text messages.

In some instances, users of the systems and methods described herein maywish to avoid interacting with certain other users in group chat orgaming environments. In previous chat systems, the banning and silencingof chat users is typically dealt with by administrators or moderators ofa chat server. Embodiments of the systems and methods described herein,however, allow users to have direct control over who is able to send theusers chat messages and/or chat contact invitations. For example, user Amay be allowed to block user B, so that user A no longer seescommunications from user B in any chat room, and/or user A no longerreceives personal chat contact (i.e., single chat) invitations from userB.

In various implementations, an alliance is a group of players in a game(e.g., a multiplayer online game) who can group together as a unit toenhance gameplay. Each alliance preferably has a chat room for itselfwhere members of the alliance can talk or send text messages to oneanother. This presents a need to block certain users from an alliancechat room at times.

FIG. 34A includes screenshots of a user interface 3400 that allows afirst user of a gaming system to block communications from a second userof the gaming system, in accordance with certain embodiments. Asdepicted, the first user selects (e.g., by tapping a touch screen) amanage settings icon 3402, which opens a settings window 3404. The firstuser then selects a “block from alliance” button 3406 on the settingswindow 3404. A message window 3408 appears informing the first user thatfuture communications from the second user will be blocked. The seconduser may be added to a list of other users who have been blocked by thefirst user. The first user may have the option of editing this list toadd or remove users to or from the list. For example, referring to FIG.34B, the next time the user selects the manage settings icon 3402, thesettings window 3404 may include an unblock from alliance button 3410.When the first user selects the unblock from alliance button 3410,future communications from the second user may be unblocked, and amessage window 3412 may appear informing the first user that suchcommunications have been unblocked.

In some instances, the complexity of the system is brought in or reducedby the scale at which blocking and unblocking is executed. Parallelcomputation may provide the flexibility to execute the blocking andunblocking at real-time, without the disadvantages of time lag seen intraditional systems. For example, parallel processing may be used totranslate and/or transform text messages in a text message chat system.A separate parallel process may be assigned to each user of a chatsession and/or each language being used in the chat session. Suchparallel processing may simplify the task of blocking and unblockingusers. For example, separate parallel processes may be removed or addedfrom the chat system as users are blocked or unblocked, respectively.

FIG. 35 is a flowchart of a method 3500 of blocking a user from a chatsession, the method 3500 includes providing (step 3502) a text messagechat system to a plurality of users of an online game. A request isreceived (step 3504) from a first user of the text message chat systemto block a second user of the text message chat system. Followingreceipt of the request, preventing (step 3506) text messages from thesecond user from being displayed for the first user. In some instances,the text messages in the chat session are translated and/or transformedusing the systems and methods described herein. Parallel processes maybe used to perform the translation and/or transformation of the textmessages. For example a separate parallel process may be assigned tohandle translation and/or transformation of text messages for eachparticular user of the chat session and/or for each language involved inthe chat session.

Automated translation services are not always accurate and may benefitoccasionally from human intervention to correct certain errors. In someimplementations, the translation systems and methods described hereinallow users to identify translation errors and offer corrections to fixthese errors. For example, a bilingual or foreign language user (e.g., aFrench player of an online game) may view a chat window and see atranslation (e.g., to or from French) that is incorrect. The user maysubmit a suggested correction for the erroneous translation, and theuser may be rewarded (e.g., with in-game currency or virtual items) forsubmitting the correction.

In certain implementations, an original text message and a correspondingtranslation are displayed on a single screen, which provides anopportunity for someone experienced in the languages to provide feedbackon the translation instantly. For example, a user may recognize atranslation error and select an option to submit a correctedtranslation. The user may then enter and submit the correctedtranslation and may receive a reward if and when the correctedtranslation is approved. Upon submitting the corrected translation, theuser may be prevented from submitting an additional correctedtranslation for the original message. A user may therefore be unable toearn multiple rewards from a single erroneous translation.

In some instances, the systems and methods are unable to translate anoriginal message because the original message was not entered correctlyby a user. For example, FIG. 36A shows an original Spanish message 3602that recites “Eres el peor!” An automated English translation 3604 ofthis message is shown in FIG. 36B and recites “You are the best!”Referring to FIGS. 36C and 36D, a user may recognize that the originalmessage was not entered in proper Spanish, which resulted in anincorrect translation. To address this error, the user may select a“correct translation button” 3606, which causes a correction window 3608to open where the user may enter a correction for the original message.Referring to FIGS. 36E and 36F, in this case, the user enters “Ustedesson los mejores!” in the correction window 3608 and selects a submitbutton 3610. A confirmation window 3612 appears informing the user thatthe submission will be processed. Closing the confirmation window 3612returns the user to the original chat page.

Referring to FIGS. 37A and 37B, a user interface 3700 is provided thatallows users to review translation corrections received from otherusers. Users who review the translation corrections may be rewarded fortheir efforts, and may be able to select the particular type of rewardthey receive (e.g., virtual goods or currency for an online game). Ingeneral, after a translation correction is submitted by a user, otherusers can decide whether the correction is better than the originaltranslation and any other translation corrections that have beensubmitted by other users. When a user's translation is judged to be thebest translation, that user may receive an award, and the user'stranslation may be added to the translation dictionary (e.g., thetranslation data store 210). Users who participate in judging thevarious translations may also receive a reward. Such rewards, however,may be given only to those users who select the translation correctionthat was chosen to be the best by all of the judges.

In general, by allowing users to submit suggested translationcorrections and to judge other users' submissions, the systems andmethods take advantage of feedback that users are willing to givefreely. The data collected in this process, once approved, may be usedto correct translation cache entries, thereby improving the overalltranslation capabilities of the systems and methods described herein.This may ensure that the correct translation is shown in the future,when the original message is submitted again for translation.

In a typical implementation, there are two types of users who may submitand/or judge translation corrections: users who are monolingual, andusers who are bilingual. Bilingual users are generally able tounderstand the original language sentence and provide a more accuratetranslation in a different language. By contrast, monolingual users maynot understand the original language phrase, but are nonetheless able toreview the translation (which is presented in the user's devicelanguage) and submit a correction in exchange for a reward. Thetranslations obtained from the two types of users tend to differ incontent, with bilingual users generally providing a more accuratetranslation. The systems and methods are preferably able to determine ordetect whether a user is monolingual or bilingual, and the user'sfeedback may be weighed according to that determination. For example,users may be able to identify themselves to the systems and methods asbeing either monolingual or bilingual.

In certain situations, most of the users are monolingual and speak thesame language (English). With a large supply of speakers of onelanguage, there are generally more users to submit translationcorrections for that language, and there is generally less demand fortranslation corrections to or from that language. To stimulate thesupply of translation corrections for other languages, users may berewarded according to the demand for translation corrections. Forexample, when a majority of users speak one language and there is noshortage of translation corrections given in that language, such usersmay receive a smaller reward (e.g., 75% of the nominal amount) forsubmitting translation corrections. At the same time, a minority ofusers who speak a different language may receive a larger reward (e.g.,125% of the nominal amount), due to a greater demand for translationcorrections in that different language.

The number of translations a user may correct over a given time period(e.g., one day) may or may not be limited. There may be no limit on thenumber of translation corrections, for example, when no reward is givenfor submitting the corrections. On the other hand, when users arerewarded such submissions, a user may be allowed to submit a limitednumber of translation corrections during the time period. Such a limitmay prevent users who are bilingual, or users who have a tendency tosubmit large numbers of translation corrections, from receivingexcessive rewards and thereby obtaining an unfair advantage in anunderlying game (e.g., a multi-player online game).

In certain instances, feedback on an incorrect translation may bereceived from only a small number of users (e.g., 2 or 3 users), whichmay make it difficult to determine correctness of translationsubmissions and to automatically generate rewards. For example, chatsoccur in a continuous stream, and many users may be more focused onchatting with other users and/or playing an underlying game, and lessfocused on submitting translation corrections. Users may also selectchats based on what they see in their window, and few users may selectthe same chat. Accordingly, when more than one translation correctionhas been received, the proposed corrections may be made available forother users to judge, in an effort to gain consensus on the correcttranslation, in exchange for rewards.

Rewards for submitting translation corrections may be given to usersaccording to a raffle system. In such a system, rewards are not givenfor every submission but may be given out randomly, with users whosubmit more corrections being more likely to earn a reward. Such anapproach reduces the likelihood that certain players may achieve anunfair advantage over other users, due to their ability and/or desire totranslate messages, rather than their ability or effort in theunderlying game.

In addition to allowing users to correct bad translations, users mayalso be able to submit feedback regarding wrongly detected languages,unfiltered profanities, and named entity detection. For example, whenviewing an original message and a translated message, a user mayrecognize that the automated translation system detected the originallanguage improperly. The user may then inform the system about thislanguage detection mistake, in exchange for a possible reward. Likewise,a user may be able to inform the system about any profanity that appearsin messages, thereby allowing the system to filter or delete suchprofanity from future messages. Users may also be able to inform thesystem about named entities, such as companies, brands, trademarks,etc., that appear in messages. This may allow the systems and methods torecognize when named entities appear in messages and to ensure suchentities are named and/or identified appropriately.

In general, translation corrections submitted by users need carefulevaluation to ensure users are rewarded only for accurate corrections.This will improve overall accuracy of the system and prevent users fromcheating by submitting fraudulent corrections. In some implementations,the accuracy of translation corrections is automatically evaluated usingword-based features, language-based features, and other features (e.g.,a word alignment match feature), to prevent users from exploiting thesystem. A part of speech (POS) based language model may be used to checksentences for grammatical correctness. Additionally, some users maysubmit translation corrections that are grammatically correct but havenothing to do with the original message. For such cases, a wordalignment match analysis feature may be useful and may be run asperiodic process to approve and/or reject user submissions. A machinelearning approach may be used to validate sparse user feedback in thetranslation systems and methods described herein.

Table 2 presents examples of suggested translation corrections submittedby users in accordance with certain embodiments of the invention. Inthese examples, the original message in the source language is “aaa bbbccc,” and the correct translation in the target language is “xxx yyyzzz.” The column labeled “Shown Translation” includes examples ofinitial translations proposed by the automated systems described herein.

TABLE 2 Examples of user corrections and preferred outcomes. OriginalShown User Message Translation Correction Description Status aaa bbb

.;$@#! xxx yyy zzz User has corrected an Ap- ccc incorrect translation.proved xxx yyy xxx yyy zzz User has improved the Ap- quality of existingproved machine translation which is partially correct. aaa xxx uuu xxx′yyy′ zzz′ User does not know Ap- target language but proved simply usesonline translation services. xxx yyy zzz xxx zzz yyy User wants toexploit Denied system for rewards by slightly rearranging words in showntranslation. xxx yyy zzz xxx yyy zzz Copy pasting the same Deniedtranslation for cheating. xxx yyy zzz

$@%{circumflex over ( )}&/ Typing a random Denied dse reyyfwfmessage/junk for cheating if copy pasted translations were denied. xxxyyy zzz sss ddd fff User submits a Denied grammatically correct messagein target language but translation is irrelevant (Copy pasting thetranslation of previous msg). xxx yyy zzz XXX yyY User attempts tosubmit Denied zzz a genuine translation but his correction is not better(poor quality) than the shown translation.

As shown in Table 2, when a user submits a correct and improvedtranslation, the user submission should be approved, and the user mayreceive an appropriate reward. When the user submits a poor quality orfraudulent translation (e.g., a random message), however, the usersubmission should be denied, and no reward should be given to the user.The systems and methods preferably approve or reject such examples asshown in the “Status” column of this table.

In certain embodiments, a translation of an original message isclassified according to whether the translation is appropriate for theoriginal message. The classification may treated as a binaryclassification task in which features are extracted from the translationand the original message. The classification technique may be used toensure translation corrections submitted by users are accurate. Forexample, in some instances, the majority-based validation describedherein is not suitable due to the small number of responses (e.g., oneto three) that may be received per incorrect translation. Theclassification technique may also be used to identify and/or addresshash collisions that appear in cached translation data. For example,about 10% or more of the translation entries in a data table may becorrupt due to hash collisions.

Referring to FIG. 38, in various implementations, the accuracy oftranslations is evaluated using a translation accuracy module 3800 thatincludes a word-based feature module 3802, a language-based featuremodule 3804, and a word alignment module 3806. The word-based featuremodule 3802 is used to assess word-based features, such as word counts,character counts, emojis, numbers, and/or punctuation marks. Forexample, when a translation is correct, the number of words in theoriginal message and the number of words in the translation aregenerally about the same. Accordingly, if the number of words in the twomessage differs by more than a threshold amount (e.g., a factor of two),the translation may be regarded as being incorrect or more likelyincorrect. In one example, if the number of words in one of the messages(e.g., the translation) is ½ (or fewer than ½) of the number of words inthe other message (e.g., the original message), the word-based featuremodule 3802 may conclude that the translation is incorrect or morelikely incorrect.

Another word-based feature that may be used to assess the accuracy of atranslation is the number of characters (e.g., letters and numbers) inthe original message and in the translation. In general, when the numberof characters the original message is about the same as the number ofcharacters in the translation, the translation is more likely to beaccurate. A threshold amount may be used to determine when the charactercounts in the two messages differ excessively. For example, if thetranslation has more than 3/2 as many characters as the originalmessage, the word-based feature module 3802 may conclude that thetranslation is incorrect or more likely incorrect.

Another word-based feature that may be used to assess the accuracy of atranslation is the count and order of emojis (e.g., ideograms or smileysused in Japanese electronic messages), which generally remain unchangedbetween the original message and the translation. Emojis tend to fallunder a certain Unicode text range that could be used to detect them ina given sentence. A regular expression may be used to identify orextract emojis from both of the messages using this Unicode range. Forexample if the input contains 3 emojis consecutively, and the outputcontains just one emoji, it indicates a disparity between input andoutput. If the count and/or order of the emojis is different between thetwo messages, the word-based feature module 3802 may conclude that thetranslation is incorrect or more likely incorrect.

An additional word-based feature that may be used to assess translationaccuracy is the count of any numbers and punctuation marks in the twomessages. For example, numbers and punctuation marks, if any, may beidentified or extracted in the original message and the translation, andthe length of the longest common subsequence (LCS) may be found betweenthem, after sorting. This length, divided by the maximum of the lengthsof the two messages, gives a real numbered value for this word-basedfeature. In general, the real numbered value provides an indication of apercentage of the numbers and punctuations in the two messages thatoverlap. Experimental results show that better results are obtainedusing a real numbered value rather than a binary value, for thisparticular feature. For example, an input sentence of “I am going tomeet you at 4:30 Cya!!” in English could have an equivalent output of“Je vais vous recontrer a 4:30 Au revoir!!.” On extracting thepunctuation and numbers we get “4:30!!” for both the English and Frenchversions. The LCS in this case would be 6 (by character) and the maximumof lengths from among the English and French versions is 36 (bycharacter). The resultant real numbered value for this word-basedfeature is 6/36=0.167.

Relying on word-based features alone may be insufficient for assessingtranslation accuracy. For example, users may be able to fool at leastsome of the word-based features by submitting a translation correctionin which each word of the original message is replaced with a dummy word(e.g., “xxx”), to produce a fraudulent correction having the same numberof words and characters present in the original message.

To avoid this issue, the translation accuracy module 3800 may use thelanguage-based feature module 3804 to evaluate language-based features,in addition to or instead of the word-based features. For example, inone embodiment, the words present in the original message and in thetranslation are tagged (e.g., using open source POS tagger) to identifyparts of speech (POS) (e.g., verbs, nouns, adjectives, etc.) in the twomessages. Each word in the messages may be tagged according to the partsof speech, using a different tag set for each language, with a differentnumber of tags. For example a sample sentence of “The device is easy touse” can be tagged by a POS tagger as “The_DT_device_NP is_VBZ easy_JJto_TO use_VB,” showing the part of speech of each word in the sentence.In this case, the tags are Determiner (DT), Noun phrase (NP), Singularpresent tense verb (VBZ), Adjective (JJ), To (TO), and Simple verb (VB).The tags of primary interest for this purpose are typically verbs,followed by adjectives and adverbs.

In certain instances, the original message and the translation aretagged separately (e.g., using POS tagger), and the resulting tags foreach message are counted to identify the number of verbs, adjectives,adverbs, etc., in each message. Due to the different types of verbs usedin each language (e.g., modal verbs, infinite verbs, past tense verbs,future tense verbs, etc.), a simplified verb tag VB may be obtainedusing a map for all types of verbs in each language. For example,English verb part of speech tags may be mapped to a single verb tag VB,as follows: ‘VBD’ (Verb, past tense)=>‘VB,’ ‘VBG’ (Verb, Gerund)=>‘VB,’‘VBN’ (Verb, past participle)=>‘VB,’ ‘VBP’ (Verb, non third personsingular present)=>‘VB,’ and ‘VBZ’ (Verb, third person singularpresent)=>‘VB.’ The POS tags in the tagged messages may be replaced withthis simplified POS tag set.

After simplifying the POS tags, the number of verb tags VB may becounted in both the original message and in the translation. Ideally,the number of verbs in each message should be the same, but there aresome exceptions. For example, “was sleeping” in English translates to“dormais” in French. The English POS tagger may tag “was” and “sleeping”as two different verbs, whereas the French POS tagger may tag “dormais”as a single verb. Verbs such as “is,” “was,” and “can” are known asauxiliary verbs in English. Other languages may not have an equivalentfor these auxiliary verbs and may instead use a single verb as areplacement. To account for such differences in verb use among thelanguages, the systems and methods may use a threshold value (e.g., 2 or3) for difference in the number of verbs between the original messageand the translation. For example, when the difference between thenumbers of verbs in the two messages is greater than two, thelanguage-based feature module 3804 may consider the translation to beincorrect or more likely incorrect. This threshold value of two wasfound to produce reasonable results through trial and error. Other partsof speech (e.g., adjectives and adverbs) may be counted and comparedbetween the two messages, using one or more additional threshold values.

In some instances, however, a user may fool this translation accuracycheck by simply copying and submitting the existing translation as acorrection for the translation. In that case, the submission may beclassified as a valid correction, but the user may not be rewarded forthe submission. In some cases, a user may also simply change the case ofsome words in the existing translation to produce and submit a validcorrection, and the user may deserve a reward and be rewarded for thesubmission. Accordingly, in some embodiments, the systems and methodsdetermine whether the existing translation and the user submission arethe same or not. If the existing translation and the user submission arethe same (e.g., including case and capitalization), no reward may begiven to the user.

In certain embodiments, the POS tags check is used to identify instanceswhen an automated translation system failed to correctly identify thelanguage of the original message. For example, the language of theoriginal message may have been detected incorrectly when a user'stranslation correction passes the word count check but fails in the POStags check. Incorrect language detection is also likely when the numberof verbs is equal to zero or all tags are nouns in one message and notin other. For example, an original Spanish message may recite: “Pizt toenviÃ³ tan pronto regrese una marcha.” If the language is detected asbeing English, however, the English POS tagger will likely be unable totag the message and, as a default, may tag all words as nouns. TheEnglish POS tagger's output may be, for example: [{“tag”: “NP”, “word”:“Pizt”, “stem”: “<unknown>”}, {“tag”: “NN”, “word”: “te”, “stem”: “te”},{“tag”: “NN”, “word”: “envi\u00c3\u00b3”, “stem”: “<unknown>”}, {“tag”:“NN”, “word”: “tan”, “stem”: “tan”}, {“tag”: “RB”, “word”: “pronto”,“stem”: “pronto”}, {“tag”: “JJ”, “word”: “regrese”, “stem”:“<unknown>”}, {“tag”: “NN”, “word”: “una”, “stem”: “<unknown>”}, {“tag”:“NN”, “word”: “marcha”, “stem”: “<unknown>”}]. By comparison, theSpanish tagger's output for the same original message may be: [{“tag”:“NP”, “word”: “Pizt”, “stem”: “<unknown>”}, {“tag”: “PPX”, “word”: “te”,“stem”: “t\u00fa”}, {“tag”: “VLfin”, “word”: “envi\u00c3\u00b3”, “stem”:“<unknown>”}, {“tag”: “ADV”, “word”: “tan”, “stem”: “tan”}, {“tag”:“ADV”, “word”: “pronto”, “stem”: “pronto”}, {“tag”: “VLfin”, “word”:“regrese”, “stem”: “regresar”}, {“tag”: “ART”, “word”: “una”, “stem”:“un”}, {“tag”: “NC”, “word”: “marcha”, “stem”: “marcha”}]. The tags“NN,” “RB,” and “PPX” refer to Noun (singular or mass), Adverb, andPossessive pronoun, respectively.

Accordingly, in certain instances, the parts of speech of the originalmessage and a translation are compared to determine whether the languagewas properly identified in the original message. In general, a languagedetection failure is more likely to have occurred when one of themessages (e.g., the translation) is tagged has having a non-zero numberof verbs and the other message (e.g., the original message) is taggedhas having no verbs. Language detection failure is also more likely whenall words in one message are tagged as nouns while the other message hasseveral types of POS tags (e.g., nouns, verbs, and adjectives).

In various embodiments, translation accuracy is assessed by identifyingand reviewing the proper nouns in the original message and in thetranslation. In general, when a translation is accurate, the propernouns (e.g., names of people and cities) are the same in translation andin the original message. Comparing the two messages and filtering commonwords that were left untranslated may therefore be useful as a featurefor identifying genuine translations. In some instances, the presence ofsuch untranslated proper nouns may help improve translation precision,but the absence of any untranslated proper nouns may not give anyinformation about translation precision. If proper nouns are identifiedin the original message but not in the translation, the accuracy of thetranslation may be considered to be incorrect or more likely incorrect.A penalty may be added to a real valued score returned for this propernoun feature, which helps identify any bad translations and improvetranslation accuracy. For example, when the proper nouns areinconsistent between the two messages, an accuracy score for thetranslation may be reduced by the penalty.

Alternatively or additionally, translation accuracy may be evaluated byanalyzing and comparing the grammar in the original message and thetranslation. Working with multiple languages may make it difficult toparse trees for all languages to understand the grammar of the sentence.The messages are also often written in chat language, which follows adifferent grammar compared to plain or formal speech in the nativelanguage.

Accordingly, to recognize a pattern among the grammar of the chatlanguage, the sentence may be tagged with POS tags to build an N-gramlanguage model on the POS tags, thereby providing an approximation ofthe underlying grammatical structure. An n-gram may be defined as acollection of n-consecutive words. A model of these n-grams may betypical for a given language and/or may be used to represent nconsecutive words in the given language. In certain implementations, themethod of word-based n-grams is extended to a Part of Speech-basedn-grams. In other words, a shallow method of parsing sentences may beused where words in a sentence are tagged with a POS tagger. In oneapproach, a BLEU score is computed on POS tags rather than on actualtext.

A trigram (3-gram) language model may be created on the POS taggedsentences for each language. For example, the sentence “The device iseasy to use” has a POS tagged output of “The_DT device_NP is_VBZ easy_JJto_TO use_VB.” Word-based trigrams in this sentence would be {The,device, is}, {device, is, easy}, {is, easy, to}, {easy, to, use}. Thecorresponding POS based trigrams would be {DT, NP, VBZ}, {NP, VBZ, JJ},{VBZ, JJ, TO}, {JJ, TO, VB}.

Trigrams with higher probabilities may be used to infer partialstructures of the grammar. For example, a trigram language model builton POS tags may have a probability associated with each trigram. Theprobability may be computed as the ratio of the number of times a giventrigram has occurred in a text corpus compared to all the trigrams inthe same text. A grammatical trigram tends to be repeated often andhence will have a higher probability (also known as a language modelscore). Hence, when a message receives a higher score on this languagemodel, the message is more likely to be grammatically correct. Thisscore may be useful to detect instances when a user types a spam messageto get rewards. The score may also be useful to determine when thelanguage detection has failed. For example, since separate models may beused for each language, the score of a sentence in the wrong languagemay be much lower. The score may also be useful for detecting when thequality of a translation is good. Separate models may be trained forhuman and machine translations for this purpose.

In certain embodiments, the language model is trained using translationsthat have been verified as being accurate. A trigram model may be builton the POS tags.

An inherent problem with n-grams of any size is a lack of all possiblehypotheses. In such cases, a backoff method is followed where n-1-gramsand n-2-grams are identified. For example, if an unknown POS trigram isseen, such as {DT, NP, JJ}, the model may retroactively check to see ifthe bigrams {DT, NP} and {NP, JJ} have a language model score indicatinggrammaticality. If the bigrams do not have probabilities (or if theprobabilities are too low), another backoff may be done to check unigramlanguage model scores for {DT}, {NP}, {JJ}. All three models can also begrouped by linear interpolation with more weights on trigram models,fewer weights on bigram models, and even fewer weights on unigrammodels.

The most frequently occurring trigrams of POS tags for chat English areas follows: <IN><VBD><PP>=1.0; <FW><NN><IN>=1.0; <DT><WP><VBP>=1.0; and<RB></S1></S2>=1.0. A score of 1.0 indicates a 100% probability that agiven trigram sequence is grammatical at all times it occurs. Ingeneral, chat language uses more pronouns beginning with “Wh,” followedby a verb, which is captured by <DT><WP><VBP>. People also tend to endchat language sentences with adverbs or adjectives, as in “You are cool”or “This is awesome,” which is captured by <RB></S1></S2>, where S1 andS2 are end of sentence tags. These trigrams may therefore be used torecognize at least some structure of the underlying grammar in eachlanguage. With the language model built for all languages of interest,the models may be saved in JSON format and may be loaded instantly atany time.

In some implementations, after a message has been POS tagged, thesystems and methods may count the number of trigrams in the message thatmatch with an existing trigram language model. Each trigram in the modelmay have a probability score associated with it and, for each trigram inthe message, the corresponding score of the trigram is identified in themodel. In some instances, if the score is higher than a threshold value(e.g., 0.05), the trigram is counted as a match. Otherwise, the trigramis not counted as a match. The systems and methods may compute thenumber of matches of trigrams to a total number of trigrams in thesentence, and this ratio may be used as real valued feature forassessing translation accuracy. For example, a trigram of {easy, to,use} which may occur frequently in grammatical text can have a highprobability score (language model score) of around 0.68. By contrast, anungrammatical trigram of {easy, use, to} could have a smallerprobability of occurrence of around 0.04. When compared with a definedthreshold of 0.05, the ungrammatical trigrams may be filtered out, andthe number of trigrams above the threshold may provide a quantitativevalue for the grammaticality of the text.

After obtaining a POS tagged message, the probability of the sentencemay be computed using the trigram language model. In one embodiment, thelog probability of each trigram in the sentence is determined, and thesum of the log probabilities is computed. This sum is then divided bythe number of words in the sentence, to obtain a score for the sentence.The score may be used as a real valued feature for assessing translationaccuracy. The log probability of a trigram is preferably calculated as alinear interpolation of trigram, bigram, and unigram probabilities. Forexample, in the sentence “The device is easy to use,” the POS taggedoutput is “The_DT device_NP is_VBZ easy_JJ to_TO use_VB.” The POS basedtrigrams for the sentence are {DT, NP, VBZ}, {NP, VBZ, JJ}, {VBZ, JJ,TO}, and {JJ, TO, VB}. Each of these trigrams has a probability ofoccurrence in a given language corpus. Assuming these probabilities are0.12, 0.44, 0.32, and 0.89 for the respective trigrams, a combined scorefor grammaticality of the sentence may be computed as a log probability.In this case, the log probability is computed aslog(0.12)+log(0.44)+log(0.32)+log(0.89), which is equal to −1.82. On arelative scale, the log probability provides a numerical indication ofthe grammaticality of the sentence.

In addition to or instead of the word-based and language-based featuresdescribed above, the translation accuracy module 3800 may use theword-alignment module 3806 to assess an alignment of words between theoriginal message and the translation. To generate a listing of potentialword alignment pairs, a translation accuracy checking algorithm was runwith word-based features alone on a database and parallel corpus wasextracted with translation pairs having probability values>0.90. Thisindicates that only good quality translation message pairs were filteredto create the parallel corpus. 100K sentences were collected for Englishand French pairs and English and Spanish pairs. The parallel corpus forthese 100K sentences was fed to a statistical machine translationtoolkit (i.e., GIZA++) to extract word alignments. The toolkit extractedapproximately 25-30K word alignment pairs and the associated probabilityscores for these pairs.

Given that there are usually multiple word alignments per word, it maybe beneficial to select only alignments that have a probability scoregreater than a certain threshold (e.g., 0.01). Even with the threshold,however, more than one alignment may be obtained per word, most of whichare typically due to spelling errors or the same word in a differenttense (e.g., past tense or future tense). Sample word alignments betweenEnglish and French are shown in Table 3. Separate word alignments may beobtained for both possible orders of two languages (e.g., English toFrench and French to English). Two word alignment files may therefore beextracted per language pair. The word alignments for the source language(i.e., the language of the original message) to the target language(i.e., the language of the translation) may be referred to as sourcealignments, while the word alignments for the target language to thesource language may be referred to as target alignments.

TABLE 3 Example word alignments between English and French. EnglishFrench Troops troupes troupe Attacks attaques attaque Smoke fumée fumefumez fumee

The source alignments and the target alignments may be loaded into twoseparate files. For each word in the original message that is alsopresent in the source alignments, the systems and methods determine ifat least one corresponding word is present in the translation and alsoin the target alignments. The same process is then applied to thetranslation sentence. Finally, the percentage of words that were foundin the word alignments file is returned as a feature for assessingtranslation accuracy.

In certain implementations, word alignments are extracted for languagepairs that include English as one of the languages. For example, thelanguage pairs may be English coupled with one of Spanish, French,Polish, Portuguese, Dutch, German, Danish, Swedish, Turkish, Italian,and Russian, for a total of 11 language pairs and 11×2=22 word alignmentfiles. For language pairs that do not include English (e.g., translatingFrench to Russian), English may be used as an intermediate language toextract alignments. For example, when validating a translation fromFrench to Russian, French to English may be used to extract wordalignments from the original message, and Russian to English may be usedto extract word alignments from the translation. The intersection ofthese two sets gives a percentage of match among the word alignments inboth messages. This method may be extended to all pairs of languages.

In various embodiments, user confidence is calculated to provide anindication of whether a user's translation submissions may be trusted.Apart from other approaches discussed herein, a user may be trusted moreor less based on the user's history of translation corrections. Thenumber of translations corrected and the number of translations rewardedmay be stored in a data table for various users. This allows apercentage of corrections done by the player to be calculated, andtranslation corrections may be approved, at least in part, based on theparticular user's confidence threshold. This can easily separate outspammers from genuine players who correct translations.

FIG. 39 is a flowchart of a method 3900 of correcting translationerrors, in accordance with certain embodiments. The method 3900 includesproviding (step 3902) a text message chat system to a plurality of usersof an online game. An original text message is received (step 3904) in afirst language from a first user of the online game. An initialtranslation is generated (step 3906) in a second language of theoriginal text message. The original text message and the initialtranslation are provided (step 3908) to a second user of the onlinegame. For example, the second user may view the two translations, eithertogether or separately, on a display of a client device. A translationcorrection is received (step 3910) from the second user to address anerror in the initial translation. The method 3900 may also includeidentifying (step 3912) a most accurate translation correction from aplurality of translation corrections (i.e., including the translationcorrection from the second user). Alternatively or additionally, themethod 3900 includes evaluating (step 3914) an accuracy of thetranslation correction from the second user using at least one of aword-based feature, a language-based feature, and a word alignmentfeature.

The word-based and language-based features described above (e.g., fourword-based features and four language-based features) may be fit using alinear regression model. After training, the model preferably returns areal valued number for each translation pair, and a threshold value maybe used to classify each translation pair as either good or bad. Forexample, if the features return numeric values of x1, x2, x3, . . . x8,a regression equation may be y=a1*x1+a2*x2++a8*x8, where a1, a2, . . .a8 are coefficients obtained from modeling a linear regression equationand y is an output value. A preferred value for the threshold value ywas found to be 0.65 after experiments with precision and recall usingan ROC curve.

After adding the word alignment based feature described above, andre-running linear regression on the word-based and language-basedfeatures, the preferred threshold was changed to 0.76. Adding the wordalignment based feature also increased the AUC value from 0.853 to0.976.

Table 4 shows coefficients of regression obtained from training 13 ksentences for each of the word-based feature, language-based features,and word alignment feature. The results in the table indicate that theemojis feature and the character counts have small coefficients ofregression, after normalization, which means these features were foundto contribute little to the assessment of translation of accuracy.

TABLE 4 Coefficients of regression for translation accuracy featuresFeature Coefficient Numbers 5.39E−01 Emojis 6.99E−20 Character Counts1.76E−17 Word Counts 2.07E−01 Count <VB> 9.55E−02 Trigram Match 1.67E−02Trigram Probs −2.24E−02   Untranslated nouns 4.18E−02 Word AlignmentMatch 4.48E−01

The regression model was evaluated by 10-fold cross validation on 13 ksentences of French to English and Spanish to English pairs. The goldstandard labels for the data were computed using BING translation withsome human supervision. Table 5 presents the precision, recall,accuracy, and F-measure values in percentages for the evaluation.Precision represents a ratio of true positive translation pairs out ofall translation pairs marked as true by our method. Recall is a ratio ofthe true positive translation pairs out of all translation pairs knownto be true pairs. Accuracy is a ratio of a sum of outcomes marked astrue positives and true negatives to the total test set size. F-measureis a harmonic mean of precision and recall. These metrics may be used inclassification tasks to measure system performance and reliability. 13 ksentences were used as a training set, and 400 hand-annotated sentenceswere used for the test. Since the test set was small, the numbers fortest were lower than the numbers for the 13 k sentences.

TABLE 5 Results from study of translation accuracy with 13,000sentences. 13k sentences 13k sentences (Word-based Measure (10fold cv)400 sentences features) Precision 90 77 88.2 Recall 79.3 72 79.3F-Measure 84.31 74.41 83.56 Accuracy 97.3 96 97

Table 6 shows the results of a 10-fold cross validation on a 13 ksentence data set, where W refers to the use of word-based features. Lrefers to the use of language-based features, and A refers to the use ofthe word alignment feature. The results show that word-based featureshelp to improve precision, and the word alignment feature helps toimprove recall significantly. Language-based features give a littleboost to both precision and recall. In general, recall indicates howaccurately bad decisions are detected from a total dataset. The resultsin the table show that adding word alignment-based features improvesrecall. Precision indicates how accurately good translations werepredicted out of the total translations marked as correct by the system.The results in the table show that adding word-based features improvesprecision.

TABLE 6 Results of 10-fold cross validation on a 13,000 sentence dataset. Method Precision Recall F-Measure Accuracy W 88.2 79.3 83.56 97 L51.9 26.6 35.17 92.5 A 55 96.7 70.11 93.9 WL 90 79.3 84.56 97.3 WA 80.596.6 87.81 97.7 LA 57.8 96.6 72.32 94.2 WLA 80.7 96.8 88.01 97.7

Table 7 shows the results of fitting the various word-based,language-based, and word alignment features with other machinealgorithms, in accordance with certain embodiments. Results until nowhave been illustrated using linear regression techniques for binding thedifferent features together. Machine Learning algorithms exist that canbe used to bind together variables (features in this context) to producean ensemble outcome that is better than the individual parts. Linearregression presents a uni-dimensional method for combining variables.Two-dimensional and multi-dimensional methods for combining variablesexist in Machine Learning algorithm literature. These algorithms wereemployed to find a more optimal way of combining the features used inthe task of predicting good translation pairs.

The results in Table 7 were obtained by combining the features withvarious Machine Learning algorithms. The dataset used included 13 ksentences, and the parameters were tuned using a gridSearch algorithm.From the methods listed in the table, the gradient boosting classifierand the random forests methods are ensemble based methods, whichexplains why these methods gave better results. Gradient BoostingMachines (GBM) and Random Forests give very good results, though GBMtook a longer time for training. Since the model needs to be trainedonly once, however, training time is largely irrelevant.

TABLE 7 Results of fitting translation accuracy features with machinealgorithms. ML Algorithm Params Precision Recall Accuracy LinearRegression (Least Squares) 80.7 96.8 97.8 Perceptron 98.8 74.3 97.8Ridge Classifier 96.1 71.6 97.3 Gaussian Naive Bayes 80.7 90.3 97.4Decision Tree 91.3 90.9 98.2 Logistic Regression L1 norm, C = 100, 92.191.4 98.4 tol = 0.001 SVM C = 1000, 91.3 92.8 98.5 gamma = 0.001Gradient Boosting n_estimators = 100 92.4 93 98.8 Machines RandomForests n_estimators = 10 93.3 92.8 98.8

The final translation accuracy checking algorithm was tested on a heldout test set of 3045 English-French sentence pairs. The results areshown in Table 8. WLAplain features perform poorly due to the fact thatword alignments were extracted from plain speak databases. Since themessages are modified after a series of transformations, a sharp drop inthe results can be found. Word alignment were extracted from chat speakdatabases and some smoothing was done for smaller sentences to avoid azero score in features. WLA shows the result for the chatspeakdatabases. Bad precision with WLA features indicate many correcttranslations were denied. The features were fit with linear regressionas Random Forest was overfitting the results. In general, these resultsmay be used as a basis for selecting a final set of features to be usedfor an algorithm. A higher accuracy is generally preferred, whereas ahigher precision rewards more true translation pairs entered by users,and a high recall ensures fewer wrong entries being misclassified ascorrect. The WLA feature set is desirable according to the results inthe table. A threshold of 0.75 may be selected for a higher recall(e.g., to obtain the fewest possible malicious entries being marked ascorrect), and a threshold of 0.68 may be selected in cases when higherprecision is desired.

TABLE 8 Results from translation accuracy checking algorithm. Area Pre-Re- F- Accu- under Features Threshold cision call Measure racy ROC WL0.75 85 59 69.65 96 0.723 WLAplain 0.75 19 79 30.63 80 0.631 WLA 0.75 6593 76.51 95 0.883 WLA 0.68 76 85 80.24 96 0.853

While the invention has been particularly shown and described withreference to specific preferred embodiments, it should be understood bythose skilled in the art that various changes in form and detail may bemade therein without departing from the spirit and scope of theinvention as defined by the appended claims.

What is claimed is:
 1. A method comprising: performing by one or more computer processors: obtaining an original text message in a first language authored by a first user; obtaining a translation of the original text message in a second language; comparing a plurality of features associated with the original text message with a respective plurality of features associated with the translation, the plurality of features comprising (i) at least one word-based feature selected from the group consisting of a word count, a character count, an emoji, a number, and a punctuation mark, and (ii) at least one language-based feature comprising a part of speech selected from group consisting of verbs, nouns, adverbs, and adjectives; calculating a metric based on the comparison of the features, and determining an accuracy of the translation based on the calculated metric.
 2. The method of claim 1, wherein the translation is authored by a second user.
 3. The method of claim 1, wherein the translation comprises a correction of a prior translation of the original text message.
 4. The method of claim 1, wherein comparing the plurality of features comprises determining a difference in the at least one word-based feature between the original text message and the translation.
 5. The method of claim 1, wherein comparing the plurality of features comprises determining a difference in a number of occurrences of the part of speech between the original text message and the translation.
 6. The method of claim 1, wherein calculating the metric comprises determining an alignment of at least one word in the original text message with at least one respective word in the translation.
 7. The method of claim 1, wherein the metric is further based on grammar of the translation.
 8. The method of claim 1, wherein calculating the metric comprises: generating a part-of-speech n-gram representation for the translation; and computing a probability of the n-gram representation for the second language.
 9. The method of claim 1, wherein calculating the metric comprises fitting the features with a linear regression model.
 10. The method of claim 1, wherein calculating the metric comprises fitting the features with one or more machine learning algorithms.
 11. A system comprising: a non-transitory computer readable medium having instructions stored thereon; and at least one processor configured to execute the instructions to perform operations comprising: obtaining an original text message in a first language authored by a first user; obtaining a translation of the original text message in a second language; comparing a plurality of features associated with the original text message with a respective plurality of features associated with the translation, the plurality of features comprising (i) at least one word-based feature selected from the group consisting of a word count, a character count, an emoji, a number, and a punctuation mark, and (ii) at least one language-based feature comprising a part of speech selected from group consisting of verbs, nouns, adverbs, and adjectives; calculating a metric based on the comparison of the features, and determining an accuracy of the translation based on the calculated metric.
 12. The system of claim 11, wherein the translation is authored by a second user.
 13. The system of claim 11, wherein the translation comprises a correction of a prior translation of the original text message.
 14. The system of claim 11, wherein comparing the plurality of features comprises determining a difference in the at least one word-based feature between the original text message and the translation.
 15. The system of claim 11, wherein comparing the plurality of features comprises determining a difference in a number of occurrences of the part of speech between the original text message and the translation.
 16. The system of claim 11, wherein calculating the metric comprises determining an alignment of at least one word in the original text message with at least one respective word in the translation.
 17. The system of claim 11, wherein the metric is further based on grammar of the translation.
 18. The system of claim 11, wherein calculating the metric comprises: generating a part-of-speech n-gram representation for the translation; and computing a probability of the n-gram representation for the second language.
 19. The system of claim 11, wherein calculating the metric comprises fitting the features with a linear regression model.
 20. The system of claim 11, wherein calculating the metric comprises fitting the features with one or more machine learning algorithms.
 21. A manufacture comprising: non-transitory computer readable media comprising executable instructions, the executable instructions being executable by one or more processors to perform operations comprising: obtaining an original text message in a first language authored by a first user; obtaining a translation of the original text message in a second language; comparing a plurality of features associated with the original text message with a respective plurality of features associated with the translation, the plurality of features comprising (i) at least one word-based feature selected from the group consisting of a word count, a character count, an emoji, a number, and a punctuation mark, and (ii) at least one language-based feature comprising a part of speech selected from group consisting of verbs, nouns, adverbs, and adjectives; calculating a metric based on the comparison of the features, and determining an accuracy of the translation based on the calculated metric. 